Synthetic CT generation from MRI using 3D transformer-based denoising diffusion model

Advancements in synthetic CT generation from MRI: A review of techniques, and trends in radiation therapy planning

BACKGROUND

Magnetic resonance imaging (MRI) and Computed tomography (CT) are crucial imaging techniques in both diagnostic imaging and radiation therapy. MRI provides excellent soft tissue contrast but lacks the direct electron density data needed to calculate dosage. CT, on the other hand, remains the gold standard due to its accurate electron density information in radiation therapy planning (RTP) but it exposes patients to ionizing radiation. Synthetic CT (sCT) generation from MRI has been a focused study field in the last few years due to cost effectiveness as well as for the objective of minimizing side-effects of using more than one imaging modality for treatment simulation. It offers significant time and cost efficiencies, bypassing the complexities of co-registration, and potentially improving treatment accuracy by minimizing registration-related errors. In an effort to navigate the quickly developing field of precision medicine, this paper investigates recent advancements in sCT generation techniques, particularly those using machine learning (ML) and deep learning (DL). The review highlights the potential of these techniques to improve the efficiency and accuracy of sCT generation for use in RTP by improving patient care and reducing healthcare costs. The intricate web of sCT generation techniques is scrutinized critically, with clinical implications and technical underpinnings for enhanced patient care revealed.

PURPOSE

This review aims to provide an overview of the most recent advancements in sCT generation from MRI with a particular focus of its use within RTP, emphasizing on techniques, performance evaluation, clinical applications, future research trends and open challenges in the field.

METHODS

A thorough search strategy was employed to conduct a systematic literature review across major scientific databases. Focusing on the past decade's advancements, this review critically examines emerging approaches introduced from 2013 to 2023 for generating sCT from MRI, providing a comprehensive analysis of their methodologies, ultimately fostering further advancement in the field. This study highlighted significant contributions, identified challenges, and provided an overview of successes within RTP. Classifying the identified approaches, contrasting their advantages and disadvantages, and identifying broad trends were all part of the review's synthesis process.

RESULTS

The review identifies various sCT generation approaches, consisting atlas-based, segmentation-based, multi-modal fusion, hybrid approaches, ML and DL-based techniques. These approaches are evaluated for image quality, dosimetric accuracy, and clinical acceptability. They are used for MRI-only radiation treatment, adaptive radiotherapy, and MR/PET attenuation correction. The review also highlights the diversity of methodologies for sCT generation, each with its own advantages and limitations. Emerging trends incorporate the integration of advanced imaging modalities including various MRI sequences like Dixon sequences, T1-weighted (T1W), T2-weighted (T2W), as well as hybrid approaches for enhanced accuracy.

CONCLUSIONS

The study examines MRI-based sCT generation, to minimize negative effects of acquiring both modalities. The study reviews 2013-2023 studies on MRI to sCT generation methods, aiming to revolutionize RTP by reducing use of ionizing radiation and improving patient outcomes. The review provides insights for researchers and practitioners, emphasizing the need for standardized validation procedures and collaborative efforts to refine methods and address limitations. It anticipates the continued evolution of techniques to improve the precision of sCT in RTP.

Exploring dual energy CT synthesis in CBCT-based adaptive radiotherapy and proton therapy: application of denoising diffusion probabilistic models

Background.Adaptive radiotherapy (ART) requires precise tissue characterization to optimize treatment plans and enhance the efficacy of radiation delivery while minimizing exposure to organs at risk. Traditional imaging techniques such as cone beam computed tomography (CBCT) used in ART settings often lack the resolution and detail necessary for accurate dosimetry, especially in proton therapy.Purpose.This study aims to enhance ART by introducing an innovative approach that synthesizes dual-energy computed tomography (DECT) images from CBCT scans using a novel 3D conditional denoising diffusion probabilistic model (DDPM) multi-decoder. This method seeks to improve dose calculations in ART planning, enhancing tissue characterization.Methods.We utilized a paired CBCT-DECT dataset from 54 head and neck cancer patients to train and validate our DDPM model. The model employs a multi-decoder Swin-UNET architecture that synthesizes high-resolution DECT images by progressively reducing noise and artifacts in CBCT scans through a controlled diffusion process.Results.The proposed method demonstrated superior performance in synthesizing DECT images (High DECT MAE 39.582 ± 0.855 and Low DECT MAE 48.540± 1.833) with significantly enhanced signal-to-noise ratio and reduced artifacts compared to traditional GAN-based methods. It showed marked improvements in tissue characterization and anatomical structure similarity, critical for precise proton and radiation therapy planning.Conclusions.This research has opened a new avenue in CBCT-CT synthesis for ART/APT by generating DECT images using an enhanced DDPM approach. The demonstrated similarity between the synthesized DECT images and ground truth images suggests that these synthetic volumes can be used for accurate dose calculations, leading to better adaptation in treatment planning.

Results of 2023 survey on the use of synthetic computed tomography for magnetic resonance Imaging-only radiotherapy: Current status and future steps

Background and purpose

The emergence of synthetic CT (sCT) in MR-guided radiotherapy (MRgRT) represents a significant advancement, supporting MR-only workflows and online treatment adaptation. However, the lack of consensus guidelines has led to varied practices. This study reports results from a 2023 ESTRO survey aimed at defining current practices in sCT development and use.

Materials and methods

An survey was distributed to ESTRO members, including 98 questions across four sections on sCT algorithm generation and usage. By June 2023, 100 centers participated. The survey revealed diverse clinical experiences and roles, with primary sCT use in the pelvis (60%), brain (15%), abdomen (11%), thorax (8%), and head-and-neck (6%). sCT was mostly used for conventional fractionation treatments (68%), photon SBRT (40%), and palliative cases (28%), with limited use in proton therapy (4%).

Results

Conditional GANs and GANs were the most used neural network architectures, operating mainly on 1.5 T and 3 T MRI images. Less than half used paired images for training, and only 20% performed image selection. Key MR image quality parameters included magnetic field homogeneity and spatial integrity. Half of the respondents lacked a dedicated sCT-QA program, and many did not apply sanitychecks before calculation. Selection strategies included age, weight, and metal artifacts. A strong consensus (95%) emerged for vendor neutral guidelines.

Conclusion

The survey highlights the need for expert-based, vendor-neutral guidelines to standardize sCT tools, metrics, and clinical protocols, ensuring effective sCT use in MR-guided radiotherapy.

Challenges and opportunities in the development and clinical implementation of artificial intelligence based synthetic computed tomography for magnetic resonance only radiotherapy

Synthetic computed tomography (sCT) generated from magnetic resonance imaging (MRI) can serve as a substitute for planning CT in radiation therapy (RT), thereby removing registration uncertainties associated with multi-modality imaging pairing, reducing costs and patient radiation exposure. CE/FDA-approved sCT solutions are nowadays available for pelvis, brain, and head and neck, while more complex deep learning (DL) algorithms are under investigation for other anatomic sites. The main challenge in achieving a widespread clinical implementation of sCT lies in the absence of consensus on sCT commissioning and quality assurance (QA), resulting in variation of sCT approaches across different hospitals. To address this issue, a group of experts gathered at the ESTRO Physics Workshop 2022 to discuss the integration of sCT solutions into clinics and report the process and its outcomes. This position paper focuses on aspects of sCT development and commissioning, outlining key elements crucial for the safe implementation of an MRI-only RT workflow.

A Deep Learning Approach for the Fast Generation of Synthetic Computed Tomography from Low-Dose Cone Beam Computed Tomography Images on a Linear Accelerator Equipped with Artificial Intelligence

Artificial Intelligence (AI) is revolutionising many aspects of radiotherapy (RT), opening scenarios that were unimaginable just a few years ago. The aim of this study is to propose a Deep Leaning (DL) approach able to quickly generate synthetic Computed Tomography (CT) images from low-dose Cone Beam CT (CBCT) acquired on a modern linear accelerator integrating AI. Methods: A total of 53 patients treated in the pelvic region were enrolled and split into training (30), validation (9), and testing (14). A Generative Adversarial Network (GAN) was trained for 200 epochs. The image accuracy was evaluated by calculating the mean and mean absolute error (ME and ME) between sCT and CT. RT treatment plans were calculated on CT and sCT images, and dose accuracy was evaluated considering Dose Volume Histogram (DVH) and gamma analysis. Results: A total of 4507 images were selected for training. The MAE and ME values in the test set were 36 ± 6 HU and 7 ± 6 HU, respectively. Mean gamma passing rates for 1%/1 mm, 2%/2 mm, and 3%/3 mm tolerance criteria were respectively 93.5 ± 3.4%, 98.0 ± 1.3%, and 99.2 ± 0.7%, with no difference between curative and palliative cases. All the DVH parameters analysed were within 1 Gy of the difference between sCT and CT. Conclusion: This study demonstrated that sCT generation using the DL approach is feasible on low-dose CBCT images. The proposed approach can represent a valid tool to speed up the online adaptive procedure and remove CT simulation from the RT workflow.

Fast MRI Reconstruction Using Deep Learning-based Compressed Sensing: A Systematic Review

Magnetic resonance imaging (MRI) has revolutionized medical imaging, providing a non-invasive and highly detailed look into the human body. However, the long acquisition times of MRI present challenges, causing patient discomfort, motion artifacts, and limiting real-time applications. To address these challenges, researchers are exploring various techniques to reduce acquisition time and improve the overall efficiency of MRI. One such technique is compressed sensing (CS), which reduces data acquisition by leveraging image sparsity in transformed spaces. In recent years, deep learning (DL) has been integrated with CS-MRI, leading to a new framework that has seen remarkable growth. DL-based CS-MRI approaches are proving to be highly effective in accelerating MR imaging without compromising image quality. This review comprehensively examines DL-based CS-MRI techniques, focusing on their role in increasing MR imaging speed. We provide a detailed analysis of each category of DL-based CS-MRI including end-to-end, unroll optimization, self-supervised, and federated learning. Our systematic review highlights significant contributions and underscores the exciting potential of DL in CS-MRI. Additionally, our systematic review efficiently summarizes key results and trends in DL-based CS-MRI including quantitative metrics, the dataset used, acceleration factors, and the progress of and research interest in DL techniques over time. Finally, we discuss potential future directions and the importance of DL-based CS-MRI in the advancement of medical imaging. To facilitate further research in this area, we provide a GitHub repository that includes up-to-date DL-based CS-MRI publications and publicly available datasets - https://github.com/mosaf/Awesome-DL-based-CS-MRI.

Synthetic CT imaging for PET monitoring in proton therapy: a simulation study

Objective.This study addresses a fundamental limitation of in-beam positron emission tomography (IB-PET) in proton therapy: the lack of direct anatomical representation in the images it produces. We aim to overcome this shortcoming by pioneering the application of deep learning techniques to create synthetic control CT images (sCT) from combining IB-PET and planning CT scan data.Approach.We conducted simulations involving six patients who underwent irradiation with proton beams. Leveraging the architecture of a visual transformer (ViT) neural network, we developed a model to generate sCT images of these patients using the planning CT scans and the inter-fractional simulated PET activity maps during irradiation. To evaluate the model's performance, a comparison was conducted between the sCT images produced by the ViT model and the authentic control CT images-serving as the benchmark.Main results.The structural similarity index was computed at a mean value across all patients of 0.91, while the mean absolute error measured 22 Hounsfield Units (HU). Root mean squared error and peak signal-to-noise ratio values were 56 HU and 30 dB, respectively. The Dice similarity coefficient exhibited a value of 0.98. These values are comparable to or exceed those found in the literature. More than 70% of the synthetic morphological changes were found to be geometrically compatible with the ones reported in the real control CT scan.Significance.Our study presents an innovative approach to surface the hidden anatomical information of IB-PET in proton therapy. Our ViT-based model successfully generates sCT images from inter-fractional PET data and planning CT scans. Our model's performance stands on par with existing models relying on input from cone beam CT or magnetic resonance imaging, which contain more anatomical information than activity maps.

Deep conditional generative model for longitudinal single-slice abdominal computed tomography harmonization

Purpose

Two-dimensional single-slice abdominal computed tomography (CT) provides a detailed tissue map with high resolution allowing quantitative characterization of relationships between health conditions and aging. However, longitudinal analysis of body composition changes using these scans is difficult due to positional variation between slices acquired in different years, which leads to different organs/tissues being captured.

Approach

To address this issue, we propose C-SliceGen, which takes an arbitrary axial slice in the abdominal region as a condition and generates a pre-defined vertebral level slice by estimating structural changes in the latent space.

Results

Our experiments on 2608 volumetric CT data from two in-house datasets and 50 subjects from the 2015 Multi-Atlas Abdomen Labeling Challenge Beyond the Cranial Vault (BTCV) dataset demonstrate that our model can generate high-quality images that are realistic and similar. We further evaluate our method's capability to harmonize longitudinal positional variation on 1033 subjects from the Baltimore longitudinal study of aging dataset, which contains longitudinal single abdominal slices, and confirmed that our method can harmonize the slice positional variance in terms of visceral fat area.

Conclusion

This approach provides a promising direction for mapping slices from different vertebral levels to a target slice and reducing positional variance for single-slice longitudinal analysis. The source code is available at: https://github.com/MASILab/C-SliceGen.