Synthetic CT generation for MRI-guided adaptive radiotherapy in prostate cancer

Challenges and opportunities in the development and clinical implementation of artificial intelligence based synthetic computed tomography for magnetic resonance only radiotherapy

Synthetic computed tomography (sCT) generated from magnetic resonance imaging (MRI) can serve as a substitute for planning CT in radiation therapy (RT), thereby removing registration uncertainties associated with multi-modality imaging pairing, reducing costs and patient radiation exposure. CE/FDA-approved sCT solutions are nowadays available for pelvis, brain, and head and neck, while more complex deep learning (DL) algorithms are under investigation for other anatomic sites. The main challenge in achieving a widespread clinical implementation of sCT lies in the absence of consensus on sCT commissioning and quality assurance (QA), resulting in variation of sCT approaches across different hospitals. To address this issue, a group of experts gathered at the ESTRO Physics Workshop 2022 to discuss the integration of sCT solutions into clinics and report the process and its outcomes. This position paper focuses on aspects of sCT development and commissioning, outlining key elements crucial for the safe implementation of an MRI-only RT workflow.

Morphometric and clinical comparison of MRI-based synthetic CT to conventional CT of the hip in children

BACKGROUND

MRI-based synthetic CT (sCT) generates CT-like images from MRI data.

OBJECTIVE

To evaluate equivalence, inter- and intraobserver reliability, and image quality of sCT compared to conventional (cCT) for assessing hip morphology and maturity in pediatric patients.

MATERIALS AND METHODS

We prospectively enrolled patients <21 years old with cCT and 3T MRI of the hips/pelvis. A dual-echo gradient-echo sequence was used to generate sCT via a commercially available post-processing software (BoneMRI v1.5 research version, MRIguidance BV, Utrecht, NL). Two pediatric musculoskeletal radiologists measured seven morphologic hip parameters. 3D surface distances between cCT and sCT were computed. Physeal status was established at seven locations with cCT as reference standard. Images were qualitatively scored on a 5-point Likert scale regarding diagnostic quality, signal-to-noise ratio, clarity of bony margin, corticomedullary differentiation, and presence and severity of artifacts. Quantitative evaluation of Hounsfield units (HU) was performed in bone, muscle, and fat tissue. Inter- and intraobserver reliability were measured by intraclass correlation coefficients. The cCT-to-sCT intermodal agreement was assessed via Bland-Altman analysis. The equivalence between modalities was tested using paired two one-sided tests. The quality parameter scores of each imaging modality were compared via Wilcoxon signed-rank test. For tissue-specific HU measurements, mean absolute error and mean percentage error values were calculated using the cCT as the reference standard.

RESULTS

Thirty-eight hips in 19 patients were included (16.6 ± 3 years, range 9.9-20.9; male = 5). cCT- and sCT-based morphologic measurements demonstrated good to excellent inter- and intraobserver correlation (0.77

CONCLUSION

sCT is equivalent to cCT for the assessment of hip morphology, physeal status, and radiodensity assessment in pediatric patients.

Collapse

Key Words

• Hip imaging
• Hip morphology
• Hounsfield units
• MRI based
• Quality
• Synthetic CT

Collapse

MESH Headings

• Humans
• Male
• Child
• Female
• Magnetic Resonance Imaging/methods
• Prospective Studies
• Reproducibility of Results
• Adolescent
• Tomography, X-Ray Computed/methods
• Hip Joint/diagnostic imaging
• Child, Preschool
• Imaging, Three-Dimensional/methods

Collapse

Grants Collapse

Affiliation(s)

Jade Iwasaka-Neder Department of Radiology, Boston Children's Hospital, 300 Longwood Ave, Boston, MA, 02115, USA.
M Alejandra Bedoya Department of Radiology, Boston Children's Hospital, 300 Longwood Ave, Boston, MA, 02115, USA
James Connors Department of Radiology, Boston Children's Hospital, 300 Longwood Ave, Boston, MA, 02115, USA
Simon Warfield Computational Radiology Laboratory, Boston Children's Hospital, 401 Park Drive, Boston, MA, 02215, USA
Sarah D Bixby Department of Radiology, Boston Children's Hospital, 300 Longwood Ave, Boston, MA, 02115, USA

Collapse

Clinical feasibility of deep learning-based synthetic CT images from T2-weighted MR images for cervical cancer patients compared to MRCAT

This work aims to investigate the clinical feasibility of deep learning-based synthetic CT images for cervix cancer, comparing them to MR for calculating attenuation (MRCAT). Patient cohort with 50 pairs of T2-weighted MR and CT images from cervical cancer patients was split into 40 for training and 10 for testing phases. We conducted deformable image registration and Nyul intensity normalization for MR images to maximize the similarity between MR and CT images as a preprocessing step. The processed images were plugged into a deep learning model, generative adversarial network. To prove clinical feasibility, we assessed the accuracy of synthetic CT images in image similarity using structural similarity (SSIM) and mean-absolute-error (MAE) and dosimetry similarity using gamma passing rate (GPR). Dose calculation was performed on the true and synthetic CT images with a commercial Monte Carlo algorithm. Synthetic CT images generated by deep learning outperformed MRCAT images in image similarity by 1.5% in SSIM, and 18.5 HU in MAE. In dosimetry, the DL-based synthetic CT images achieved 98.71% and 96.39% in the GPR at 1% and 1 mm criterion with 10% and 60% cut-off values of the prescription dose, which were 0.9% and 5.1% greater GPRs over MRCAT images.

A systematic literature review: deep learning techniques for synthetic medical image generation and their applications in radiotherapy

The aim of this systematic review is to determine whether Deep Learning (DL) algorithms can provide a clinically feasible alternative to classic algorithms for synthetic Computer Tomography (sCT). The following categories are presented in this study: ∙ MR-based treatment planning and synthetic CT generation techniques. ∙ Generation of synthetic CT images based on Cone Beam CT images. ∙ Low-dose CT to High-dose CT generation. ∙ Attenuation correction for PET images. To perform appropriate database searches, we reviewed journal articles published between January 2018 and June 2023. Current methodology, study strategies, and results with relevant clinical applications were analyzed as we outlined the state-of-the-art of deep learning based approaches to inter-modality and intra-modality image synthesis. This was accomplished by contrasting the provided methodologies with traditional research approaches. The key contributions of each category were highlighted, specific challenges were identified, and accomplishments were summarized. As a final step, the statistics of all the cited works from various aspects were analyzed, which revealed that DL-based sCTs have achieved considerable popularity, while also showing the potential of this technology. In order to assess the clinical readiness of the presented methods, we examined the current status of DL-based sCT generation.

CT synthesis from MR images using frequency attention conditional generative adversarial network

Magnetic resonance (MR) image-guided radiotherapy is widely used in the treatment planning of malignant tumors, and MR-only radiotherapy, a representative of this technique, requires synthetic computed tomography (sCT) images for effective radiotherapy planning. Convolutional neural networks (CNN) have shown remarkable performance in generating sCT images. However, CNN-based models tend to synthesize more low-frequency components and the pixel-wise loss function usually used to optimize the model can result in blurred images. To address these problems, a frequency attention conditional generative adversarial network (FACGAN) is proposed in this paper. Specifically, a frequency cycle generative model (FCGM) is designed to enhance the inter-mapping between MR and CT and extract more rich tissue structure information. Additionally, a residual frequency channel attention (RFCA) module is proposed and incorporated into the generator to enhance its ability in perceiving the high-frequency image features. Finally, high-frequency loss (HFL) and cycle consistency high-frequency loss (CHFL) are added to the objective function to optimize the model training. The effectiveness of the proposed model is validated on pelvic and brain datasets and compared with state-of-the-art deep learning models. The results show that FACGAN produces higher-quality sCT images while retaining clearer and richer high-frequency texture information.

Artificial intelligence applications in prostate cancer

Artificial intelligence (AI) applications have enabled remarkable advancements in healthcare delivery. These AI tools are often aimed to improve accuracy and efficiency of histopathology assessment and diagnostic imaging interpretation, risk stratification (i.e., prognostication), and prediction of therapeutic benefit for personalized treatment recommendations. To date, multiple AI algorithms have been explored for prostate cancer to address automation of clinical workflow, integration of data from multiple domains in the decision-making process, and the generation of diagnostic, prognostic, and predictive biomarkers. While many studies remain within the pre-clinical space or lack validation, the last few years have witnessed the emergence of robust AI-based biomarkers validated on thousands of patients, and the prospective deployment of clinically-integrated workflows for automated radiation therapy design. To advance the field forward, multi-institutional and multi-disciplinary collaborations are needed in order to prospectively implement interoperable and accountable AI technology routinely in clinic.

Deep learning based synthesis of MRI, CT and PET: Review and analysis

Medical image synthesis represents a critical area of research in clinical decision-making, aiming to overcome the challenges associated with acquiring multiple image modalities for an accurate clinical workflow. This approach proves beneficial in estimating an image of a desired modality from a given source modality among the most common medical imaging contrasts, such as Computed Tomography (CT), Magnetic Resonance Imaging (MRI), and Positron Emission Tomography (PET). However, translating between two image modalities presents difficulties due to the complex and non-linear domain mappings. Deep learning-based generative modelling has exhibited superior performance in synthetic image contrast applications compared to conventional image synthesis methods. This survey comprehensively reviews deep learning-based medical imaging translation from 2018 to 2023 on pseudo-CT, synthetic MR, and synthetic PET. We provide an overview of synthetic contrasts in medical imaging and the most frequently employed deep learning networks for medical image synthesis. Additionally, we conduct a detailed analysis of each synthesis method, focusing on their diverse model designs based on input domains and network architectures. We also analyse novel network architectures, ranging from conventional CNNs to the recent Transformer and Diffusion models. This analysis includes comparing loss functions, available datasets and anatomical regions, and image quality assessments and performance in other downstream tasks. Finally, we discuss the challenges and identify solutions within the literature, suggesting possible future directions. We hope that the insights offered in this survey paper will serve as a valuable roadmap for researchers in the field of medical image synthesis.

Environmental Sustainability and AI in Radiology: A Double-Edged Sword

According to the World Health Organization, climate change is the single biggest health threat facing humanity. The global health care system, including medical imaging, must manage the health effects of climate change while at the same time addressing the large amount of greenhouse gas (GHG) emissions generated in the delivery of care. Data centers and computational efforts are increasingly large contributors to GHG emissions in radiology. This is due to the explosive increase in big data and artificial intelligence (AI) applications that have resulted in large energy requirements for developing and deploying AI models. However, AI also has the potential to improve environmental sustainability in medical imaging. For example, use of AI can shorten MRI scan times with accelerated acquisition times, improve the scheduling efficiency of scanners, and optimize the use of decision-support tools to reduce low-value imaging. The purpose of this Radiology in Focus article is to discuss this duality at the intersection of environmental sustainability and AI in radiology. Further discussed are strategies and opportunities to decrease AI-related emissions and to leverage AI to improve sustainability in radiology, with a focus on health equity. Co-benefits of these strategies are explored, including lower cost and improved patient outcomes. Finally, knowledge gaps and areas for future research are highlighted.

A study of Bayesian deep network uncertainty and its application to synthetic CT generation for MR-only radiotherapy treatment planning

BACKGROUND

The use of synthetic computed tomography (CT) for radiotherapy treatment planning has received considerable attention because of the absence of ionizing radiation and close spatial correspondence to source magnetic resonance (MR) images, which have excellent tissue contrast. However, in an MR-only environment, little effort has been made to examine the quality of synthetic CT images without using the original CT images.

PURPOSE

To estimate synthetic CT quality without referring to original CT images, this study established the relationship between synthetic CT uncertainty and Bayesian uncertainty, and proposed a new Bayesian deep network for generating synthetic CT images and estimating synthetic CT uncertainty for MR-only radiotherapy treatment planning.

METHODS AND MATERIALS

A novel deep Bayesian network was formulated using probabilistic network weights. Two mathematical expressions were proposed to quantify the Bayesian uncertainty of the network and synthetic CT uncertainty, which was closely related to the mean absolute error (MAE) in Hounsfield Unit (HU) of synthetic CT. These uncertainties were examined to demonstrate the accuracy of representing the synthetic CT uncertainty using a Bayesian counterpart. We developed a hybrid Bayesian architecture and a new data normalization scheme, enabling the Bayesian network to generate both accurate synthetic CT and reliable uncertainty information when probabilistic weights were applied. The proposed method was evaluated in 59 patients (13/12/32/2 for training/validation/testing/uncertainty visualization) diagnosed with prostate cancer, who underwent same-day pelvic CT- and MR-acquisitions. To assess the relationship between Bayesian and synthetic CT uncertainties, linear and non-linear correlation coefficients were calculated on per-voxel, per-tissue, and per-patient bases. For accessing the accuracy of the CT number and dosimetric accuracy, the proposed method was compared with a commercially available atlas-based method (MRCAT) and a U-Net conditional-generative adversarial network (UcGAN).

RESULTS

The proposed model exhibited 44.33 MAE, outperforming UcGAN 52.51 and MRCAT 54.87. The gamma rate (2%/2 mm dose difference/distance to agreement) of the proposed model was 98.68%, comparable to that of UcGAN (98.60%) and MRCAT (98.56%). The per-patient and per-tissue linear correlation coefficients between the Bayesian and synthetic CT uncertainties ranged from 0.53 to 0.83, implying a moderate to strong linear correlation. Per-voxel correlation coefficients varied from -0.13 to 0.67 depending on the regions-of-interest evaluated, indicating tissue-dependent correlation. The R2 value for estimating MAE solely using Bayesian uncertainty was 0.98, suggesting that the uncertainty of the proposed model was an ideal candidate for predicting synthetic CT error, without referring to the original CT.

CONCLUSION

This study established a relationship between the Bayesian model uncertainty and synthetic CT uncertainty. A novel Bayesian deep network was proposed to generate a synthetic CT and estimate its uncertainty. Various metrics were used to thoroughly examine the relationship between the uncertainties of the proposed Bayesian model and the generated synthetic CT. Compared with existing approaches, the proposed model showed comparable CT number and dosimetric accuracies. The experiments showed that the proposed Bayesian model was capable of producing accurate synthetic CT, and was an effective indicator of the uncertainty and error associated with synthetic CT in MR-only workflows.

MRI-Guided Radiation Therapy-An Emerging and Disruptive Process of Care: Healthcare Economic and Policy Considerations

MRI-guided radiation therapy (MRgRT) is an emerging, innovative technology that provides opportunities to transform and improve the current clinical care process in radiation oncology. As with many new technologies in radiation oncology, careful evaluation from a healthcare economic and policy perspective is required for its successful implementation. In this review article, we describe the current evidence surrounding MRgRT, framing it within the context of value within the healthcare system. Additionally, we highlight areas in which MRgRT may disrupt the current process of care, and discuss the evidence thresholds and timeline required for the widespread adoption of this promising technology.

A multi-centric evaluation of self-learning GAN based pseudo-CT generation software for low field pelvic magnetic resonance imaging

Purpose/objectives

An artificial intelligence-based pseudo-CT from low-field MR images is proposed and clinically evaluated to unlock the full potential of MRI-guided adaptive radiotherapy for pelvic cancer care.

Materials and method

In collaboration with TheraPanacea (TheraPanacea, Paris, France) a pseudo-CT AI-model was generated using end-to-end ensembled self-supervised GANs endowed with cycle consistency using data from 350 pairs of weakly aligned data of pelvis planning CTs and TrueFisp-(0.35T)MRIs. The image accuracy of the generated pCT were evaluated using a retrospective cohort involving 20 test cases coming from eight different institutions (US: 2, EU: 5, AS: 1) and different CT vendors. Reconstruction performance was assessed using the organs at risk used for treatment. Concerning the dosimetric evaluation, twenty-nine prostate cancer patients treated on the low field MR-Linac (ViewRay) at Montpellier Cancer Institute were selected. Planning CTs were non-rigidly registered to the MRIs for each patient. Treatment plans were optimized on the planning CT with a clinical TPS fulfilling all clinical criteria and recalculated on the warped CT (wCT) and the pCT. Three different algorithms were used: AAA, AcurosXB and MonteCarlo. Dose distributions were compared using the global gamma passing rates and dose metrics.

Results

The observed average scaled (between maximum and minimum HU values of the CT) difference between the pCT and the planning CT was 33.20 with significant discrepancies across organs. Femoral heads were the most reliably reconstructed (4.51 and 4.77) while anal canal and rectum were the less precise ones (63.08 and 53.13). Mean gamma passing rates for 1%1mm, 2%/2mm, and 3%/3mm tolerance criteria and 10% threshold were greater than 96%, 99% and 99%, respectively, regardless the algorithm used. Dose metrics analysis showed a good agreement between the pCT and the wCT. The mean relative difference were within 1% for the target volumes (CTV and PTV) and 2% for the OARs.

Conclusion

This study demonstrated the feasibility of generating clinically acceptable an artificial intelligence-based pseudo CT for low field MR in pelvis with consistent image accuracy and dosimetric results.

Artificial general intelligence for radiation oncology

The emergence of artificial general intelligence (AGI) is transforming radiation oncology. As prominent vanguards of AGI, large language models (LLMs) such as GPT-4 and PaLM 2 can process extensive texts and large vision models (LVMs) such as the Segment Anything Model (SAM) can process extensive imaging data to enhance the efficiency and precision of radiation therapy. This paper explores full-spectrum applications of AGI across radiation oncology including initial consultation, simulation, treatment planning, treatment delivery, treatment verification, and patient follow-up. The fusion of vision data with LLMs also creates powerful multimodal models that elucidate nuanced clinical patterns. Together, AGI promises to catalyze a shift towards data-driven, personalized radiation therapy. However, these models should complement human expertise and care. This paper provides an overview of how AGI can transform radiation oncology to elevate the standard of patient care in radiation oncology, with the key insight being AGI's ability to exploit multimodal clinical data at scale.

Machine Learning for Medical Image Translation: A Systematic Review

BACKGROUND

CT scans are often the first and only form of brain imaging that is performed to inform treatment plans for neurological patients due to its time- and cost-effective nature. However, MR images give a more detailed picture of tissue structure and characteristics and are more likely to pick up abnormalities and lesions. The purpose of this paper is to review studies which use deep learning methods to generate synthetic medical images of modalities such as MRI and CT.

METHODS

A literature search was performed in March 2023, and relevant articles were selected and analyzed. The year of publication, dataset size, input modality, synthesized modality, deep learning architecture, motivations, and evaluation methods were analyzed.

RESULTS

A total of 103 studies were included in this review, all of which were published since 2017. Of these, 74% of studies investigated MRI to CT synthesis, and the remaining studies investigated CT to MRI, Cross MRI, PET to CT, and MRI to PET. Additionally, 58% of studies were motivated by synthesizing CT scans from MRI to perform MRI-only radiation therapy. Other motivations included synthesizing scans to aid diagnosis and completing datasets by synthesizing missing scans.

CONCLUSIONS

Considerably more research has been carried out on MRI to CT synthesis, despite CT to MRI synthesis yielding specific benefits. A limitation on medical image synthesis is that medical datasets, especially paired datasets of different modalities, are lacking in size and availability; it is therefore recommended that a global consortium be developed to obtain and make available more datasets for use. Finally, it is recommended that work be carried out to establish all uses of the synthesis of medical scans in clinical practice and discover which evaluation methods are suitable for assessing the synthesized images for these needs.

Synthetic computed tomography for low-field magnetic resonance-only radiotherapy in head-and-neck cancer using residual vision transformers

Background and purpose

Synthetic computed tomography (sCT) scans are necessary for dose calculation in magnetic resonance (MR)-only radiotherapy. While deep learning (DL) has shown remarkable performance in generating sCT scans from MR images, research has predominantly focused on high-field MR images. This study presents the first implementation of a DL model for sCT generation in head-and-neck (HN) cancer using low-field MR images. Specifically, the use of vision transformers (ViTs) was explored.

Materials and methods

The dataset consisted of 31 patients, resulting in 196 pairs of deformably-registered computed tomography (dCT) and MR scans. The latter were obtained using a balanced steady-state precession sequence on a 0.35T scanner. Residual ViTs were trained on 2D axial, sagittal, and coronal slices, respectively, and the final sCTs were generated by averaging the models' outputs. Different image similarity metrics, dose volume histogram (DVH) deviations, and gamma analyses were computed on the test set (n = 6). The overlap between auto-contours on sCT scans and manual contours on MR images was evaluated for different organs-at-risk using the Dice score.

Results

The median [range] value of the test mean absolute error was 57 [37-74] HU. DVH deviations were below 1% for all structures. The median gamma passing rates exceeded 94% in the 2%/2mm analysis (threshold = 90%). The median Dice scores were above 0.7 for all organs-at-risk.

Conclusions

The clinical applicability of DL-based sCT generation from low-field MR images in HN cancer was proved. High sCT-dCT similarity and dose metric accuracy were achieved, and sCT suitability for organs-at-risk auto-delineation was shown.

Research progress on deep learning in magnetic resonance imaging-based diagnosis and treatment of prostate cancer: a review on the current status and perspectives

Multiparametric magnetic resonance imaging (mpMRI) has emerged as a first-line screening and diagnostic tool for prostate cancer, aiding in treatment selection and noninvasive radiotherapy guidance. However, the manual interpretation of MRI data is challenging and time-consuming, which may impact sensitivity and specificity. With recent technological advances, artificial intelligence (AI) in the form of computer-aided diagnosis (CAD) based on MRI data has been applied to prostate cancer diagnosis and treatment. Among AI techniques, deep learning involving convolutional neural networks contributes to detection, segmentation, scoring, grading, and prognostic evaluation of prostate cancer. CAD systems have automatic operation, rapid processing, and accuracy, incorporating multiple sequences of multiparametric MRI data of the prostate gland into the deep learning model. Thus, they have become a research direction of great interest, especially in smart healthcare. This review highlights the current progress of deep learning technology in MRI-based diagnosis and treatment of prostate cancer. The key elements of deep learning-based MRI image processing in CAD systems and radiotherapy of prostate cancer are briefly described, making it understandable not only for radiologists but also for general physicians without specialized imaging interpretation training. Deep learning technology enables lesion identification, detection, and segmentation, grading and scoring of prostate cancer, and prediction of postoperative recurrence and prognostic outcomes. The diagnostic accuracy of deep learning can be improved by optimizing models and algorithms, expanding medical database resources, and combining multi-omics data and comprehensive analysis of various morphological data. Deep learning has the potential to become the key diagnostic method in prostate cancer diagnosis and treatment in the future.

Deep Learning in MRI-guided Radiation Therapy: A Systematic Review

MRI-guided radiation therapy (MRgRT) offers a precise and adaptive approach to treatment planning. Deep learning applications which augment the capabilities of MRgRT are systematically reviewed. MRI-guided radiation therapy offers a precise, adaptive approach to treatment planning. Deep learning applications which augment the capabilities of MRgRT are systematically reviewed with emphasis placed on underlying methods. Studies are further categorized into the areas of segmentation, synthesis, radiomics, and real time MRI. Finally, clinical implications, current challenges, and future directions are discussed.