MR-Only Brain Radiation Therapy: Dosimetric Evaluation of Synthetic CTs Generated by a Dilated Convolutional Neural Network

Deep Learning-Based Synthetic CT for Low Field Brain MR-Guided Radiation Therapy

PURPOSE

MR-guided Radiation Therapy (MRgRT) enables online adaptation to address intra- and inter-fractional changes. To address the need of high-fidelity synthetic CT (synCT) required for dose calculation, we developed a conditional generative adversarial network (cGAN) for synCT generation from low-field MRI in the brain.

METHODS AND MATERIALS

Simulation MR-CT pairs from twelve glioma patients imaged with a head and neck surface coil and treated on a 0.35T MR-linac were prospectively included to train the model consisting of a 9-block residual network generator and a PatchGAN discriminator. Four-fold cross validation was implemented. SynCT was quantitatively evaluated against real CT using mean absolute error (MAE), Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM). Dose was calculated on synCT applying original treatment plan. Dosimetric performance was evaluated by dose-volume histogram (DVH) metric comparison and local three-dimensional gamma analysis. To demonstrate utilization in treatment adaptation, longitudinal synCTs were generated for qualitative evaluation, and one offline adaptation case underwent two comparative plan evaluations. Secondary validation was conducted with 9 patients on a different MR-linac using a high-resolution brain coil.

RESULTS

Our model generated high-quality synCTs with MAE, PSNR and SSIM of 70.9±10.4 HU, 28.4±1.5 d.B. and 0.87±0.02 within the field-of-view, respectively. Underrepresented post-surgical anomalies challenged model performance. Nevertheless, excellent dosimetric agreement was observed with the mean difference between real and synCT DVH metrics of -0.07±0.29 Gy for target D95 and within [-0.14, 0.02] Gy for organs at risk. Significant differences were only observed in the right lens D0.01cc with negligible overall difference (<0.13 Gy). Mean gamma analysis pass rates were 92.2%±3.0%, 99.2%±0.7% and 99.9%±0.1% at 1%/1mm, 2%/2mm and 3%/3mm, respectively. Secondary validation yielded no significant differences in synCT performance for whole brain MAE, PSNR, and SSIM with comparable dosimetric results.

CONCLUSIONS

Our cGAN model generated high-fidelity brain synCTs from low-field MRI with excellent dosimetric performance. Secondary validation suggests great promise of implementing synCTs to facilitate robust dose calculation for online adaptive brain MRgRT.

Technical note: MR image-based synthesis CT for CyberKnife robotic stereotactic radiosurgery

The purpose of this study is to investigate whether deep learning-based sCT images enable accurate dose calculation in CK robotic stereotactic radiosurgery. A U-net convolutional neural network was trained using 2446 MR-CT pairs and used it to translate 551 MR images to sCT images for testing. The sCT of CK patient was encapsulated into a quality assurance (QA) validation phantom for dose verification. The CT value difference between CT and sCT was evaluated using mean absolute error (MAE) and the statistical significance of dose differences between CT and sCT was tested using the Wilcoxon signed rank test. For all CK patients, the MAE value of the whole brain region did not exceed 25 HU. The percentage dose difference between CT and sCT was less than ±0.4% on GTV (D2(Gy), -0.29%, D95(Gy), -0.09%), PTV (D2(Gy), -0.25%, D95(Gy), -0.10%), and brainstem (max dose(Gy), 0.31%). The percentage dose difference between CT and sCT for most regions of interest (ROIs) was no more than ±0.04%. This study extended MR-based sCT prediction to CK robotic stereotactic radiosurgery, expanding the application scenarios of MR-only radiation therapy. The results demonstrated the remarkable accuracy of dose calculation on sCT for patients treated with CK robotic stereotactic radiosurgery.

A unified generation-registration framework for improved MR-based CT synthesis in proton therapy

BACKGROUND

The use of magnetic resonance (MR) imaging for proton therapy treatment planning is gaining attention as a highly effective method for guidance. At the core of this approach is the generation of computed tomography (CT) images from MR scans. However, the critical issue in this process is accurately aligning the MR and CT images, a task that becomes particularly challenging in frequently moving body areas, such as the head-and-neck. Misalignments in these images can result in blurred synthetic CT (sCT) images, adversely affecting the precision and effectiveness of the treatment planning.

PURPOSE

This study introduces a novel network that cohesively unifies image generation and registration processes to enhance the quality and anatomical fidelity of sCTs derived from better-aligned MR images.

METHODS

The approach synergizes a generation network (G) with a deformable registration network (R), optimizing them jointly in MR-to-CT synthesis. This goal is achieved by alternately minimizing the discrepancies between the generated/registered CT images and their corresponding reference CT counterparts. The generation network employs a UNet architecture, while the registration network leverages an implicit neural representation (INR) of the displacement vector fields (DVFs). We validated this method on a dataset comprising 60 head-and-neck patients, reserving 12 cases for holdout testing.

RESULTS

Compared to the baseline Pix2Pix method with MAE 124.95 ± $\pm$ 30.74 HU, the proposed technique demonstrated 80.98 ± $\pm$ 7.55 HU. The unified translation-registration network produced sharper and more anatomically congruent outputs, showing superior efficacy in converting MR images to sCTs. Additionally, from a dosimetric perspective, the plan recalculated on the resulting sCTs resulted in a remarkably reduced discrepancy to the reference proton plans.

CONCLUSIONS

This study conclusively demonstrates that a holistic MR-based CT synthesis approach, integrating both image-to-image translation and deformable registration, significantly improves the precision and quality of sCT generation, particularly for the challenging body area with varied anatomic changes between corresponding MR and CT.

Clinical implementation of a commercial synthetic computed tomography solution for radiotherapy treatment of glioblastoma

Background and Purpose

Magnetic resonance (MR)-only radiotherapy (RT) workflow eliminates uncertainties due to computed tomography (CT)-MR image registration, by using synthetic CT (sCT) images generated from MR. This study describes the clinical implementation process, from retrospective commissioning to prospective validation stage of a commercial artificial intelligence (AI)-based sCT product. Evaluation of the dosimetric performance of the sCT is presented, with emphasis on the impact of voxel size differences between image modalities.

Materials and methods

sCT performance was assessed in glioblastoma RT planning. Dose differences for 30 patients in both commissioning and validation cohorts were calculated at various dose-volume-histogram (DVH) points for target and organs-at-risk (OAR). A gamma analysis was conducted on regridded image plans. Quality assurance (QA) guidelines were established based on commissioning phase results.

Results

Mean dose difference to target structures was found to be within ± 0.7 % regardless of image resolution and cohort. OARs' mean dose differences were within ± 1.3 % for plans calculated on regridded images for both cohorts, while differences were higher for plans with original voxel size, reaching up to -4.2 % for chiasma D2% in the commissioning cohort. Gamma passing rates for the brain structure using the criteria 1 %/1mm, 2 %/2mm and 3 %/3mm were 93.6 %/99.8 %/100 % and 96.6 %/99.9 %/100 % for commissioning and validation cohorts, respectively.

Conclusions

Dosimetric outcomes in both commissioning and validation stages confirmed sCT's equivalence to CT. The large patient cohort in this study aided in establishing a robust QA program for the MR-only workflow, now applied in glioblastoma RT at our center.

A systematic literature review: deep learning techniques for synthetic medical image generation and their applications in radiotherapy

The aim of this systematic review is to determine whether Deep Learning (DL) algorithms can provide a clinically feasible alternative to classic algorithms for synthetic Computer Tomography (sCT). The following categories are presented in this study: ∙ MR-based treatment planning and synthetic CT generation techniques. ∙ Generation of synthetic CT images based on Cone Beam CT images. ∙ Low-dose CT to High-dose CT generation. ∙ Attenuation correction for PET images. To perform appropriate database searches, we reviewed journal articles published between January 2018 and June 2023. Current methodology, study strategies, and results with relevant clinical applications were analyzed as we outlined the state-of-the-art of deep learning based approaches to inter-modality and intra-modality image synthesis. This was accomplished by contrasting the provided methodologies with traditional research approaches. The key contributions of each category were highlighted, specific challenges were identified, and accomplishments were summarized. As a final step, the statistics of all the cited works from various aspects were analyzed, which revealed that DL-based sCTs have achieved considerable popularity, while also showing the potential of this technology. In order to assess the clinical readiness of the presented methods, we examined the current status of DL-based sCT generation.

Contrast-enhanced to non-contrast-enhanced image translation to exploit a clinical data warehouse of T1-weighted brain MRI

BACKGROUND

Clinical data warehouses provide access to massive amounts of medical images, but these images are often heterogeneous. They can for instance include images acquired both with or without the injection of a gadolinium-based contrast agent. Harmonizing such data sets is thus fundamental to guarantee unbiased results, for example when performing differential diagnosis. Furthermore, classical neuroimaging software tools for feature extraction are typically applied only to images without gadolinium. The objective of this work is to evaluate how image translation can be useful to exploit a highly heterogeneous data set containing both contrast-enhanced and non-contrast-enhanced images from a clinical data warehouse.

METHODS

We propose and compare different 3D U-Net and conditional GAN models to convert contrast-enhanced T1-weighted (T1ce) into non-contrast-enhanced (T1nce) brain MRI. These models were trained using 230 image pairs and tested on 77 image pairs from the clinical data warehouse of the Greater Paris area.

RESULTS

Validation using standard image similarity measures demonstrated that the similarity between real and synthetic T1nce images was higher than between real T1nce and T1ce images for all the models compared. The best performing models were further validated on a segmentation task. We showed that tissue volumes extracted from synthetic T1nce images were closer to those of real T1nce images than volumes extracted from T1ce images.

CONCLUSION

We showed that deep learning models initially developed with research quality data could synthesize T1nce from T1ce images of clinical quality and that reliable features could be extracted from the synthetic images, thus demonstrating the ability of such methods to help exploit a data set coming from a clinical data warehouse.

"sCT-Feasibility" - a feasibility study for deep learning-based MRI-only brain radiotherapy

BACKGROUND

Radiotherapy (RT) is an important treatment modality for patients with brain malignancies. Traditionally, computed tomography (CT) images are used for RT treatment planning whereas magnetic resonance imaging (MRI) images are used for tumor delineation. Therefore, MRI and CT need to be registered, which is an error prone process. The purpose of this clinical study is to investigate the clinical feasibility of a deep learning-based MRI-only workflow for brain radiotherapy, that eliminates the registration uncertainty through calculation of a synthetic CT (sCT) from MRI data.

METHODS

A total of 54 patients with an indication for radiation treatment of the brain and stereotactic mask immobilization will be recruited. All study patients will receive standard therapy and imaging including both CT and MRI. All patients will receive dedicated RT-MRI scans in treatment position. An sCT will be reconstructed from an acquired MRI DIXON-sequence using a commercially available deep learning solution on which subsequent radiotherapy planning will be performed. Through multiple quality assurance (QA) measures and reviews during the course of the study, the feasibility of an MRI-only workflow and comparative parameters between sCT and standard CT workflow will be investigated holistically. These QA measures include feasibility and quality of image guidance (IGRT) at the linear accelerator using sCT derived digitally reconstructed radiographs in addition to potential dosimetric deviations between the CT and sCT plan. The aim of this clinical study is to establish a brain MRI-only workflow as well as to identify risks and QA mechanisms to ensure a safe integration of deep learning-based sCT into radiotherapy planning and delivery.

DISCUSSION

Compared to CT, MRI offers a superior soft tissue contrast without additional radiation dose to the patients. However, up to now, even though the dosimetrical equivalence of CT and sCT has been shown in several retrospective studies, MRI-only workflows have still not been widely adopted. The present study aims to determine feasibility and safety of deep learning-based MRI-only radiotherapy in a holistic manner incorporating the whole radiotherapy workflow.

TRIAL REGISTRATION

NCT06106997.

Bony structure enhanced synthetic CT generation using Dixon sequences for pelvis MR-only radiotherapy

BACKGROUND

MRI-only radiotherapy planning (MROP) is beneficial to patients by avoiding MRI/CT registration errors, simplifying the radiation treatment simulation workflow and reducing exposure to ionizing radiation. MRI is the primary imaging modality for soft tissue delineation. Treatment planning CTs (i.e., CT simulation scan) are redundant if a synthetic CT (sCT) can be generated from the MRI to provide the patient positioning and electron density information. Unsupervised deep learning (DL) models like CycleGAN are widely used in MR-to-sCT conversion, when paired patient CT and MR image datasets are not available for model training. However, compared to supervised DL models, they cannot guarantee anatomic consistency, especially around bone.

PURPOSE

The purpose of this work was to improve the sCT accuracy generated from MRI around bone for MROP.

METHODS

To generate more reliable bony structures on sCT images, we proposed to add bony structure constraints in the unsupervised CycleGAN model's loss function and leverage Dixon constructed fat and in-phase (IP) MR images. Dixon images provide better bone contrast than T2-weighted images as inputs to a modified multi-channel CycleGAN. A private dataset with a total of 31 prostate cancer patients were used for training (20) and testing (11).

RESULTS

We compared model performance with and without bony structure constraints using single- and multi-channel inputs. Among all the models, multi-channel CycleGAN with bony structure constraints had the lowest mean absolute error, both inside the bone and whole body (50.7 and 145.2 HU). This approach also resulted in the highest Dice similarity coefficient (0.88) of all bony structures compared with the planning CT.

CONCLUSION

Modified multi-channel CycleGAN with bony structure constraints, taking Dixon-constructed fat and IP images as inputs, can generate clinically suitable sCT images in both bone and soft tissue. The generated sCT images have the potential to be used for accurate dose calculation and patient positioning in MROP radiation therapy.

Clinical feasibility of a commercially available MRI-only method for radiotherapy treatment planning of the brain

BACKGROUND

Advancements in deep-learning based synthetic computed tomography (sCT) image conversion methods have enabled the development of magnetic resonance imaging (MRI)-only based radiotherapy treatment planning (RTP) of the brain.

PURPOSE

This study evaluates the clinical feasibility of a commercial, deep-learning based MRI-only RTP method with respect to dose calculation and patient positioning verification performance in RTP of the brain.

METHODS

Clinical validation of dose calculation accuracy was performed by a retrospective evaluation for 25 glioma and 25 brain metastasis patients. Dosimetric and image quality of the studied MRI-only RTP method was evaluated by a direct comparison of the sCT-based and computed tomography (CT)-based external beam radiation therapy (EBRT) images and treatment plans. Patient positioning verification accuracy of sCT images was evaluated retrospectively for 10 glioma and 10 brain metastasis patients based on clinical cone-beam computed tomography (CBCT) imaging.

RESULTS

An average mean dose difference of Dmean = 0.1% for planning target volume (PTV) and 0.6% for normal tissue (NT) structures were obtained for glioma patients. Respective results for brain metastasis patients were Dmean = 0.5% for PTVs and Dmean =1.0% for NTs. Global three-dimensional (3D) gamma pass rates using 2%/2 mm dose difference and distance-to-agreement (DTA) criterion were 98.0% for the glioma subgroup, and 95.2% for the brain metastasis subgroup using 1%/1 mm criterion. Mean distance differences of <1.0 mm were observed in all Cartesian directions between CT-based and sCT-based CBCT patient positioning in both subgroups.

CONCLUSIONS

In terms of dose calculation and patient positioning accuracy, the studied MRI-only method demonstrated its clinical feasibility for RTP of the brain. The results encourage the use of the studied method as part of a routine clinical workflow.

Artificial Intelligence in Gynaecological Malignancies: Perspectives of a Clinical Oncologist

Gynecological malignancies are treated with a multi-disciplinary approach. There are two important factors, the stage and Karnofsky performance status scale (KPS) of patients, which guide the treatment strategy by single or multiple modalities in terms of surgery, radiotherapy, or chemotherapy. Various aspects are included in the workflow of gynecological malignancies, like screening, diagnosis in each individual, treatment modalities, and finally, follow-up to see for outcomes leading to the development of new research protocols. The quality data plays an important role in every step. Artificial Intelligence (AI) will play an important role if it is developed in the above-mentioned steps. AI is already established partially in every aspect of the management of gynecological cancer. It needs to be strengthened and incorporated further in a more robust form. This needs an association between clinicians, software engineers, and stakeholders. This article reviews the role of AI in various steps of the workflow of gynecological malignancies and discusses a few clinical aspects that may be researched to find solutions by AI.

Synthetic CTs for MRI-only brain RT treatment: integration of immobilization systems

PURPOSE

Auxiliary devices such as immobilization systems should be considered in synthetic CT (sCT)-based treatment planning (TP) for MRI-only brain radiotherapy (RT). A method for auxiliary device definition in the sCT is introduced, and its dosimetric impact on the sCT-based TP is addressed.

METHODS

T1-VIBE DIXON was acquired in an RT setup. Ten datasets were retrospectively used for sCT generation. Silicone markers were used to determine the auxiliary devices' relative position. An auxiliary structure template (AST) was created in the TP system and placed manually on the MRI. Various RT mask characteristics were simulated in the sCT and investigated by recalculating the CT-based clinical plan on the sCT. The influence of auxiliary devices was investigated by creating static fields aimed at artificial planning target volumes (PTVs) in the CT and recalculated in the sCT. The dose covering 50% of the PTV (D50) deviation percentage between CT-based/recalculated plan (∆D50[%]) was evaluated.

RESULTS

Defining an optimal RT mask yielded a ∆D50[%] of 0.2 ± 1.03% for the PTV and between -1.6 ± 3.4% and 1.1 ± 2.0% for OARs. Evaluating each static field, the largest ∆D50[%] was delivered by AST positioning inaccuracy (max: 3.5 ± 2.4%), followed by the RT table (max: 3.6 ± 1.2%) and the RT mask (max: 3.0 ± 0.8% [anterior], 1.6 ± 0.4% [rest]). No correlation between ∆D50[%] and beam depth was found for the sum of opposing beams, except for (45° + 315°).

CONCLUSION

This study evaluated the integration of auxiliary devices and their dosimetric influence on sCT-based TP. The AST can be easily integrated into the sCT-based TP. Further, we found that the dosimetric impact was within an acceptable range for an MRI-only workflow.

Compensation cycle consistent generative adversarial networks (Comp-GAN) for synthetic CT generation from MR scans with truncated anatomy

BACKGROUND

MR scans used in radiotherapy can be partially truncated due to the limited field of view (FOV), affecting dose calculation accuracy in MR-based radiation treatment planning.

PURPOSE

We proposed a novel Compensation-cycleGAN (Comp-cycleGAN) by modifying the cycle-consistent generative adversarial network (cycleGAN), to simultaneously create synthetic CT (sCT) images and compensate the missing anatomy from the truncated MR images.

METHODS

Computed tomography (CT) and T1 MR images with complete anatomy of 79 head-and-neck patients were used for this study. The original MR images were manually cropped 10-25 mm off at the posterior head to simulate clinically truncated MR images. Fifteen patients were randomly chosen for testing and the rest of the patients were used for model training and validation. Both the truncated and original MR images were used in the Comp-cycleGAN training stage, which enables the model to compensate for the missing anatomy by learning the relationship between the truncation and known structures. After the model was trained, sCT images with complete anatomy can be generated by feeding only the truncated MR images into the model. In addition, the external body contours acquired from the CT images with full anatomy could be an optional input for the proposed method to leverage the additional information of the actual body shape for each test patient. The mean absolute error (MAE) of Hounsfield units (HU), peak signal-to-noise ratio (PSNR), and structural similarity index (SSIM) were calculated between sCT and real CT images to quantify the overall sCT performance. To further evaluate the shape accuracy, we generated the external body contours for sCT and original MR images with full anatomy. The Dice similarity coefficient (DSC) and mean surface distance (MSD) were calculated between the body contours of sCT and original MR images for the truncation region to assess the anatomy compensation accuracy.

RESULTS

The average MAE, PSNR, and SSIM calculated over test patients were 93.1 HU/91.3 HU, 26.5 dB/27.4 dB, and 0.94/0.94 for the proposed Comp-cycleGAN models trained without/with body-contour information, respectively. These results were comparable with those obtained from the cycleGAN model which is trained and tested on full-anatomy MR images, indicating the high quality of the sCT generated from truncated MR images by the proposed method. Within the truncated region, the mean DSC and MSD were 0.85/0.89 and 1.3/0.7 mm for the proposed Comp-cycleGAN models trained without/with body contour information, demonstrating good performance in compensating the truncated anatomy.

CONCLUSIONS

We developed a novel Comp-cycleGAN model that can effectively create sCT with complete anatomy compensation from truncated MR images, which could potentially benefit the MRI-based treatment planning.

Dosimetric feasibility of direct post-operative MR-Linac-based stereotactic radiosurgery for resection cavities of brain metastases

BACKGROUND

Post-operative radiosurgery (SRS) of brain metastases patients is typically planned on a post-recovery MRI, 2-4 weeks after resection. However, the intracranial metastasis may (re-)grow in this period. Planning SRS directly on the post-operative MRI enables shortening this time interval, anticipating the start of adjuvant systemic therapy, and so decreasing the chance of extracranial progression. The MRI-Linac (MRL) allows the simultaneous execution of the post-operative MRI and SRS treatment. The aim of this work was investigating the dosimetric feasibility of MRL-based post-operative SRS.

METHODS

MRL treatments based on the direct post-operative MRI were simulated, including thirteen patients with resectable single brain metastases. The gross tumor volume (GTV) was contoured on the direct post-operative scans and compared to the post-recovery MRI GTV. Three plans for each patient were created: a non-coplanar VMAT CT-Linac plan (ncVMAT) and a coplanar IMRT MRL plan (cIMRT) on the direct post-operative MRI, and a ncVMAT plan on the post-recovery MRI as the current clinical standard.

RESULTS

Between the direct post-operative and post-recovery MRI, 15.5 % of the cavities shrunk by > 2 cc, and 46 % expanded by ≥ 2 cc. Although the direct post-operative cIMRT plans had a higher median gradient index (3.6 vs 2.7) and median V3Gy of the skin (18.4 vs 1.1 cc) compared to ncVMAT plans, they were clinically acceptable.

CONCLUSION

Direct post-operative MRL-based SRS for resection cavities of brain metastases is dosimetrically acceptable, with the advantages of increased patient comfort and logistics. Clinical benefit of this workflow should be investigated given the dosimetric plausibility.

Adoption of Hybrid MRI-Linac Systems for the Treatment of Brain Tumors: A Systematic Review of the Current Literature Regarding Clinical and Technical Features

BACKGROUND

Possible advantages of magnetic resonance (MR)-guided radiation therapy (MRgRT) for the treatment of brain tumors include improved definition of treatment volumes and organs at risk (OARs) that could allow margin reductions, resulting in limited dose to the OARs and/or dose escalation to target volumes. Recently, hybrid systems integrating a linear accelerator and an magnetic resonance imaging (MRI) scan (MRI-linacs, MRL) have been introduced, that could potentially lead to a fully MRI-based treatment workflow.

METHODS

We performed a systematic review of the published literature regarding the adoption of MRL for the treatment of primary or secondary brain tumors (last update November 3, 2022), retrieving a total of 2487 records; after a selection based on title and abstracts, the full text of 74 articles was analyzed, finally resulting in the 52 papers included in this review.

RESULTS AND DISCUSSION

Several solutions have been implemented to achieve a paradigm shift from CT-based radiotherapy to MRgRT, such as the management of geometric integrity and the definition of synthetic CT models that estimate electron density. Multiple sequences have been optimized to acquire images with adequate quality with on-board MR scanner in limited times. Various sophisticated algorithms have been developed to compensate the impact of magnetic field on dose distribution and calculate daily adaptive plans in a few minutes with satisfactory dosimetric parameters for the treatment of primary brain tumors and cerebral metastases. Dosimetric studies and preliminary clinical experiences demonstrated the feasibility of treating brain lesions with MRL.

CONCLUSIONS

The adoption of an MRI-only workflow is feasible and could offer several advantages for the treatment of brain tumors, including superior image quality for lesions and OARs and the possibility to adapt the treatment plan on the basis of daily MRI. The growing body of clinical data will clarify the potential benefit in terms of toxicity and response to treatment.

A meta-fusion RCNN network for endoscopic visual bladder lesions intelligent detection

This study investigates a visual object detection technology in order to help doctors diagnose bladder lesions with endoscopy. A new object detection approach based on deep learning is presented, which derived from the cascade R-CNN and extended the ability of network for adapting insufficient endoscopic lesions samples when training a deep neural network. We propose a feature adaptive fusion model to increase the network's mobility and reduce the possibility of overfitting problems, and use task adaptation meta-learning approach to train the feature fusion process of the entire model and the target network update process in order to complete the task-adaptive classification and detection. The new model has been evaluated on the challenging object detection data set Pascal VOC and its converted format of Microsoft COCO, and the results show that the performance of our proposed method is superior to the original method. Therefore, we apply the proposed method to a custom bladder lesions data set to solve the auxiliary detection problem in the intelligent diagnosis of bladder lesions and demonstrated the effectiveness.

Application of Mobile Virtual Reality Technology Combined with Neural Network in Facial Expression Recognition

In recent years, in the field of virtual reality, in more and more scenes, users interact with hardware or programs through facial expressions. In order to give full play to the advantages of program interaction between virtual reality devices and users, this paper proposes a mobile virtual reality expression recognition system combined with convolution neural network. Based on the optimized AlexNet network, an expression recognition algorithm is constructed and combined with LBP feature mapping technology to improve the performance of the algorithm. At the same time, according to the nature and characteristics of mobile virtual reality devices, the user face information acquisition algorithm is optimized. The performance test results of the expression recognition system show that the recognition accuracy of the system is higher than that of the traditional convolution neural network expression recognition algorithm, and the maximum difference is greater than 10%. At the same time, the average running speed of the whole system is about 37 ms, which can meet the accuracy and real-time requirements of expression recognition in virtual reality interaction. The experimental results show that the expression recognition system proposed in this paper can be applied to mobile virtual reality devices. At the same time, it also provides new ideas for industry researchers to optimize the identification function.

Machine learning in neuro-oncology: toward novel development fields

PURPOSE

Artificial Intelligence (AI) involves several and different techniques able to elaborate a large amount of data responding to a specific planned outcome. There are several possible applications of this technology in neuro-oncology.

METHODS

We reviewed, according to PRISMA guidelines, available studies adopting AI in different fields of neuro-oncology including neuro-radiology, pathology, surgery, radiation therapy, and systemic treatments.

RESULTS

Neuro-radiology presented the major number of studies assessing AI. However, this technology is being successfully tested also in other operative settings including surgery and radiation therapy. In this context, AI shows to significantly reduce resources and costs maintaining an elevated qualitative standard. Pathological diagnosis and development of novel systemic treatments are other two fields in which AI showed promising preliminary data.

CONCLUSION

It is likely that AI will be quickly included in some aspects of daily clinical practice. Possible applications of these techniques are impressive and cover all aspects of neuro-oncology.

Personalized modeling to improve pseudo-CT images for magnetic resonance imaging-guided adaptive radiotherapy

PURPOSE

Magnetic resonance imaging-guided adaptive radiotherapy (MRIgART) greatly improves daily tumor localization and enables online re-planning to obtain maximum dosimetric benefits. However, accurately predicting patient-specific electron density maps for adaptive radiotherapy (ART) planning remains a challenge. Therefore, this study proposes a personalized modeling framework for generating pseudo-computed tomography (pCT) in MRIgART.

METHODS AND MATERIALS

Eighty-three patients who received MRIgART were included and CT simulations were performed on all the patients. Daily T2-weighted 1.5 T MRI was acquired using the Unity MR-linac for adaptive planning. Pairs of co-registered CT and daily MRI images of the randomly selected training set (68 patients) were inputted into a generative adversarial network (GAN) to establish a population model. The personalized model for each patient in the test set (15 patients) was acquired using model fine-tuning, which adopted the pair of the deformable-registered CT and the first daily MRI to fine-tune the population model. The pCT quality was quantitatively evaluated in the second and the last fractions with three metrics: intensity accuracy using mean absolute error (MAE); anatomical structure similarity using dice similarity coefficient (DSC); and dosimetric consistency using gamma-passing rate (GPR).

RESULTS

The image generation speed was 65 slices per second. For the last fractions, and for head-neck, thoracoabdominal, and pelvic cases, the average MAEs were 76.8 HU vs. 123.6 HU, 38.1 HU vs. 52.0 HU, and 29.5 HU vs. 39.7 HU, respectively. Furthermore, the average DSCs of bone were 0.92 vs. 0.80, 0.85 vs. 0.73, and 0.94 vs. 0.88; and the average GPRs (1%/1 mm) were 95.5% vs. 84.7%, 97.7% vs. 92.8%, and 95.5% vs. 88.7%, for personalized vs. population models, respectively. Results of the second fractions were similar.

CONCLUSIONS

The proposed personalized modeling framework remarkably improved pCT quality for multiple treatment sites and was well suited for the MRIgART clinical setting.

Prospective Clinical Feasibility Study for MRI-Only Brain Radiotherapy

OBJECTIVES

MRI-only radiotherapy (RT) provides a workflow to decrease the geometric uncertainty introduced by the image registration process between MRI and CT data and to streamline the RT planning. Despite the recent availability of validated synthetic CT (sCT) methods for the head region, there are no clinical implementations reported for brain tumors. Based on a preceding validation study of sCT, this study aims to investigate MRI-only brain RT through a prospective clinical feasibility study with endpoints for dosimetry and patient setup.

MATERIAL AND METHODS

Twenty-one glioma patients were included. MRI Dixon images were used to generate sCT images using a CE-marked deep learning-based software. RT treatment plans were generated based on MRI delineated anatomical structures and sCT for absorbed dose calculations. CT scans were acquired but strictly used for sCT quality assurance (QA). Prospective QA was performed prior to MRI-only treatment approval, comparing sCT and CT image characteristics and calculated dose distributions. Additional retrospective analysis of patient positioning and dose distribution gamma evaluation was performed.

RESULTS

Twenty out of 21 patients were treated using the MRI-only workflow. A single patient was excluded due to an MRI artifact caused by a hemostatic substance injected near the target during surgery preceding radiotherapy. All other patients fulfilled the acceptance criteria. Dose deviations in target were within ±1% for all patients in the prospective analysis. Retrospective analysis yielded gamma pass rates (2%, 2 mm) above 99%. Patient positioning using CBCT images was within ± 1 mm for registrations with sCT compared to CT.

CONCLUSION

We report a successful clinical study of MRI-only brain radiotherapy, conducted using both prospective and retrospective analysis. Synthetic CT images generated using the CE-marked deep learning-based software were clinically robust based on endpoints for dosimetry and patient positioning.

An MRI sequence independent convolutional neural network for synthetic head CT generation in proton therapy

A magnetic resonance imaging (MRI) sequence independent deep learning technique was developed and validated to generate synthetic computed tomography (sCT) scans for MR guided proton therapy. 47 meningioma patients previously undergoing proton therapy based on pencil beam scanning were divided into training (33), validation (6), and test (8) cohorts. T₁, T₂, and contrast enhanced T₁ (T1CM) MRI sequences were used in combination with the planning CT (pCT) data to train a 3D U-Net architecture with ResNet-Blocks. A hyperparameter search was performed including two loss functions, two group sizes of normalisation, and depth of the network. Training outcome was compared between models trained for each individual MRI sequence and for all sequences combined. The performance was evaluated based on a metric and dosimetric analysis as well as spot difference maps. Furthermore, the influence of immobilisation masks that are not visible on MRIs was investigated. Based on the hyperparameter search, the final model was trained with fixed features per group for the group normalisation, six down-convolution steps, an input size of 128×192×192, and feature loss. For the test dataset for body/bone the mean absolute error (MAE) values were on average 79.8/216.3Houndsfield unit (HU) when trained using T1 images, 71.1/186.1HU for T2, and 82.9/236.4HU for T1CM. The structural similarity metric (SSIM) ranged from 0.95 to 0.98 for all sequences. The investigated dose parameters of the target structures agreed within 1% between original proton treatment plans and plans recalculated on sCTs. The spot difference maps had peaks at ±0.2cm and for 98% of all spots the difference was less than 1cm. A novel MRI sequence independent sCT generator was developed, which suggests that the training phase of neural networks can be disengaged from specific MRI acquisition protocols. In contrast to previous studies, the patient cohort consisted exclusively of actual proton therapy patients (i.e. "real-world data").

Artificial intelligence-based bone-enhanced magnetic resonance image-a computed tomography/magnetic resonance image composite image modality in nasopharyngeal carcinoma radiotherapy

Background

In the radiotherapy of nasopharyngeal carcinoma (NPC), magnetic resonance imaging (MRI) is widely used to delineate tumor area more accurately. While MRI offers the higher soft tissue contrast, patient positioning and couch correction based on bony image fusion of computed tomography (CT) is also necessary. There is thus an urgent need to obtain a high image contrast between bone and soft tissue to facilitate target delineation and patient positioning for NPC radiotherapy. In this paper, our aim is to develop a novel image conversion between the CT and MRI modalities to obtain clear bone and soft tissue images simultaneously, here called bone-enhanced MRI (BeMRI).

Methods

Thirty-five patients were retrospectively selected for this study. All patients underwent clinical CT simulation and 1.5T MRI within the same week in Shenzhen Second People's Hospital. To synthesize BeMRI, two deep learning networks, U-Net and CycleGAN, were constructed to transform MRI to synthetic CT (sCT) images. Each network used 28 patients' images as the training set, while the remaining 7 patients were used as the test set (~1/5 of all datasets). The bone structure from the sCT was then extracted by the threshold-based method and embedded in the corresponding part of the MRI image to generate the BeMRI image. To evaluate the performance of these networks, the following metrics were applied: mean absolute error (MAE), structural similarity index (SSIM), and peak signal-to-noise ratio (PSNR).

Results

In our experiments, both deep learning models achieved good performance and were able to effectively extract bone structure from MRI. Specifically, the supervised U-Net model achieved the best results with the lowest overall average MAE of 125.55 (P<0.05) and produced the highest SSIM of 0.89 and PSNR of 23.84. These results indicate that BeMRI can display bone structure in higher contrast than conventional MRI.

Conclusions

A new image modality BeMRI, which is a composite image of CT and MRI, was proposed. With high image contrast of both bone structure and soft tissues, BeMRI will facilitate tumor localization and patient positioning and eliminate the need to frequently check between separate MRI and CT images during NPC radiotherapy.

MRI-based synthetic CT of the lumbar spine: Geometric measurements for surgery planning in comparison with CT

PURPOSE

MRI is the imaging modality of choice for soft tissue-related spine disease. However, CT is superior to MRI in providing clear visualization of bony morphology. The purpose of this study is to test equivalency of MRI-based synthetic CT to conventional CT in quantitatively assessing bony morphology of the lumbar spine.

METHOD

A prospective study with an equivalency design was performed. Adult patients who had undergone MRI and CT of the lumbar spine were included. Synthetic CT images were generated from MRI using a deep learning-based image synthesis method. Two readers independently measured pedicle width, spinal canal width, neuroforamen length, anterior and posterior vertebral body height, superior and inferior vertebral body length, superior and inferior vertebral body width, maximal disc height, lumbar curvature and spinous process length on synthetic CT and CT. The agreement among CT and synthetic CT was evaluated using equivalency statistical testing.

RESULTS

Thirty participants were included (14 men and 16 women, range 20-60 years). The measurements performed on synthetic CT of pedicle width, spinal canal width, vertebral body height, vertebral body width, vertebral body length and spinous process length were statistically equivalent to CT measurements at the considered margins. Excellent inter- and intra-reader reliability was found for both synthetic CT and CT.

CONCLUSIONS

Equivalency of MRI-based synthetic CT to CT was demonstrated on geometrical measurements in the lumbar spine. In combination with the soft tissue information of the conventional MRI, this provides new possibilities in diagnosis and surgical planning without ionizing radiation.

Deep learning methods to generate synthetic CT from MRI in radiotherapy: A literature review

PURPOSE

In radiotherapy, MRI is used for target volume and organs-at-risk delineation for its superior soft-tissue contrast as compared to CT imaging. However, MRI does not provide the electron density of tissue necessary for dose calculation. Several methods of synthetic-CT (sCT) generation from MRI data have been developed for radiotherapy dose calculation. This work reviewed deep learning (DL) sCT generation methods and their associated image and dose evaluation, in the context of MRI-based dose calculation.

METHODS

We searched the PubMed and ScienceDirect electronic databases from January 2010 to March 2021. For each paper, several items were screened and compiled in figures and tables.

RESULTS

This review included 57 studies. The DL methods were either generator-only based (45% of the reviewed studies), or generative adversarial network (GAN) architecture and its variants (55% of the reviewed studies). The brain and pelvis were the most commonly investigated anatomical localizations (39% and 28% of the reviewed studies, respectively), and more rarely, the head-and-neck (H&N) (15%), abdomen (10%), liver (5%) or breast (3%). All the studies performed an image evaluation of sCTs with a diversity of metrics, with only 36 studies performing dosimetric evaluations of sCT.

CONCLUSIONS

The median mean absolute errors were around 76 HU for the brain and H&N sCTs and 40 HU for the pelvis sCTs. For the brain, the mean dose difference between the sCT and the reference CT was <2%. For the H&N and pelvis, the mean dose difference was below 1% in most of the studies. Recent GAN architectures have advantages compared to generator-only, but no superiority was found in term of image or dose sCT uncertainties. Key challenges of DL-based sCT generation methods from MRI in radiotherapy is the management of movement for abdominal and thoracic localizations, the standardization of sCT evaluation, and the investigation of multicenter impacts.

3D MRI with CT-like bone contrast - An overview of current approaches and practical clinical implementation

CT is the imaging modality of choice for assessment of 3D bony morphology but incurs the penalty of ionizing radiation. Improving the ability of 3D MRI to provide high-resolution images of cortical bone with CT-like bone contrast has been a focus of recent research. The ability of 3D MRI to deliver cortical bone information with similar diagnostic performance to CT would complement assessment of soft tissues and medullary bone from a single MRI examination, simplifying evaluation and obviating radiation exposure from additional CT. This article presents an overview of current 3D MRI approaches for imaging cortical bone with CT-like bone contrast including ultrashort echo time, zero echo time, T1-weighted gradient recalled echo, susceptibility-weighted imaging and deep learning techniques. We also discuss clinical implementation of an optimized stack-of-stars 3D gradient recalled echo pulse sequence (3D-Bone) on commercially available MRI scanners for rendering 3D MRI with CT-like bone contrast in our institutional practice.

Deep learning based synthetic-CT generation in radiotherapy and PET: A review

Recently,deep learning (DL)-based methods for the generation of synthetic computed tomography (sCT) have received significant research attention as an alternative to classical ones. We present here a systematic review of these methods by grouping them into three categories, according to their clinical applications: (i) to replace computed tomography in magnetic resonance (MR) based treatment planning, (ii) facilitate cone-beam computed tomography based image-guided adaptive radiotherapy, and (iii) derive attenuation maps for the correction of positron emission tomography. Appropriate database searching was performed on journal articles published between January 2014 and December 2020. The DL methods' key characteristics were extracted from each eligible study, and a comprehensive comparison among network architectures and metrics was reported. A detailed review of each category was given, highlighting essential contributions, identifying specific challenges, and summarizing the achievements. Lastly, the statistics of all the cited works from various aspects were analyzed, revealing the popularity and future trends and the potential of DL-based sCT generation. The current status of DL-based sCT generation was evaluated, assessing the clinical readiness of the presented methods.

Using neural networks to extend cropped medical images for deformable registration among images with differing scan extents

PURPOSE

Missing or discrepant imaging volume is a common challenge in deformable image registration (DIR). To minimize the adverse impact, we train a neural network to synthesize cropped portions of head and neck CT's and then test its use in DIR.

METHODS

Using a training dataset of 409 head and neck CT's, we trained a generative adversarial network to take in a cropped 3D image and output an image with synthesized anatomy in the cropped region. The network used a 3D U-Net generator along with Visual Geometry Group (VGG) deep feature losses. To test our technique, for each of the 53 test volumes, we used Elastix to deformably register combinations of a randomly cropped, full, and synthetically full volume to a single cropped, full, and synthetically full target volume. We additionally tested our method's robustness to crop extent by progressively increasing the amount of cropping, synthesizing the missing anatomy using our network, and then performing the same registration combinations. Registration performance was measured using 95% Hausdorff distance across 16 contours.

RESULTS

We successfully trained a network to synthesize missing anatomy in superiorly and inferiorly cropped images. The network can estimate large regions in an incomplete image, far from the cropping boundary. Registration using our estimated full images was not significantly different from registration using the original full images. The average contour matching error for full image registration was 9.9 mm, whereas our method was 11.6, 12.1, and 13.6 mm for synthesized-to-full, full-to-synthesized, and synthesized-to-synthesized registrations, respectively. In comparison, registration using the cropped images had errors of 31.7 mm and higher. Plotting the registered image contour error as a function of initial preregistered error shows that our method is robust to registration difficulty. Synthesized-to-full registration was statistically independent of cropping extent up to 18.7 cm superiorly cropped. Synthesized-to-synthesized registration was nearly independent, with a -0.04 mm of change in average contour error for every additional millimeter of cropping.

CONCLUSIONS

Different or inadequate in scan extent is a major cause of DIR inaccuracies. We address this challenge by training a neural network to complete cropped 3D images. We show that with image completion, the source of DIR inaccuracy is eliminated, and the method is robust to varying crop extent.

Assessment of deep learning-based PET attenuation correction frameworks in the sinogram domain

This study set out to investigate various deep learning frameworks for PET attenuation correction in the sinogram domain. Different models for both time-of-flight (TOF) and non-TOF PET emission data were implemented, including direct estimation of the attenuation corrected (AC) emission sinograms from the nonAC sinograms, estimation of the attenuation correction factors (ACFs) from PET emission data, correction of scattered photons prior to training of the models, and separate training of the models for each segment of the emission sinograms. A segmentation-based 2-class AC map was included as a bottom-line technique for comparison of the different models considering PET/CT AC as reference. Fifty clinical TOF PET/CT brain scans were employed for training whereas 20 were used for evaluation of the models. Quantitative analysis of the resulting PET images was carried out through region-wise standardized uptake value (SUV) bias calculation. The models relying on TOF information significantly outperformed the nonTOF models as well as the segmentation-based AC map resulting in maximum SUV bias of 6.5%, 9.5%, and 14.0%, respectively. Estimation of ACFs from either TOF or nonTOF PET emission data was very sensitive to prior scatter correction. However, direct estimation of AC sinograms from nonAC sinograms revealed no sensitivity to scatter correction, thus obviating the need for prior scatter estimation. For TOF PET data, though direct prediction of the AC sinograms does not require prior estimation of scattered photons, it requires input/output channels equal to the number of TOF bins which might be computationally or memory-wise expensive. Prediction of the ACF matrices from TOF emission data is less demanding in terms of memory as it requires only a single channel for output. AC in the sinogram domain of TOF PET data exhibited superior performance compared to both nonTOF and segmentation-based methods. However, such models require multiple input/output channels.

Cone beam CT based validation of neural network generated synthetic CTs for radiotherapy in the head region

PURPOSE

In the past years, many different neural network-based conversion techniques for synthesizing computed tomographys (sCTs) from MR images have been published. While the model's performance can be checked during the training against the test set, test datasets can never represent the whole population. Conversion errors can still occur for special cases, for example, for unusual anatomical situations. Therefore, the performance of sCT conversion needs to be verified on a patient specific level, especially in the absence of a planning CT (pCT). In this study, the capability of cone-beam CTs (CBCTs) for the validation of sCTs generated by a neural network was investigated.

METHODS

41 patients with tumors in the head region were selected. 20 of them were used for model training and 10 for validation. Different implementations of CycleGAN (with/without identity and feature loss) were used to generate sCTs. The pixel (MAE, RMSE, PSNR) and geometric error (DICE, Sensitivity, Specificity) values were reported to identify the best model. VMAT plans were created for the remaining 11 patients on the pCTs. These plans were re-calculated on sCTs and CBCTs. An automatic density overriding method ( C B C T RS ) and a population-based dose calculation method ( C B C T Pop ) were employed for CBCT-based dose calculation. The dose distributions were analysed using 3D global gamma analysis, applying a threshold of 10% with respect to the prescribed dose. Differences in DVH metrics for the PTV and the organs-at-risk were compared among the dose distributions based on pCTs, sCTs, and CBCTs.

RESULTS

The best model was the CycleGAN without identity and feature matching loss. Including the identity loss led to a metric decrease of 10% for DICE and a metric increase of 20-60 HU for MAE. Using the 2%/2 mm gamma criterion and pCT as reference, the mean gamma pass rates were 99.0  ±  0.4% for sCTs. Mean gamma pass rate values comparing pCT and CBCT were 99.0  ±  0.8% and 99.1  ±  0.8% for the C B C T RS and C B C T Pop , respectively. The mean gamma pass rates comparing sCT and CBCT resulted in 98.4  ±  1.6% and 99.2  ±  0.6% for C B C T RS and C B C T Pop , respectively. The differences between the gamma-pass-rates of the sCT and two CBCT-based methods were not significant. The majority of deviations of the investigated DVH metrices between sCTs and CBCTs were within 2%.

CONCLUSION

The dosimetric results demonstrate good agreement between sCT, CBCT, and pCT based calculations. A properly applied CBCT conversion method can serve as a tool for quality assurance procedures in an MR only radiotherapy workflow for head patients. Dosimetric deviations of DVH metrics between sCT and CBCTs of larger than 2% should be followed up. A systematic shift of approximately 1% should be taken into account when using the C B C T RS approach in an MR only workflow.

Synthetic CT generation from weakly paired MR images using cycle-consistent GAN for MR-guided radiotherapy

Although MR-guided radiotherapy (MRgRT) is advancing rapidly, generating accurate synthetic CT (sCT) from MRI is still challenging. Previous approaches using deep neural networks require large dataset of precisely co-registered CT and MRI pairs that are difficult to obtain due to respiration and peristalsis. Here, we propose a method to generate sCT based on deep learning training with weakly paired CT and MR images acquired from an MRgRT system using a cycle-consistent GAN (CycleGAN) framework that allows the unpaired image-to-image translation in abdomen and thorax. Data from 90 cancer patients who underwent MRgRT were retrospectively used. CT images of the patients were aligned to the corresponding MR images using deformable registration, and the deformed CT (dCT) and MRI pairs were used for network training and testing. The 2.5D CycleGAN was constructed to generate sCT from the MRI input. To improve the sCT generation performance, a perceptual loss that explores the discrepancy between high-dimensional representations of images extracted from a well-trained classifier was incorporated into the CycleGAN. The CycleGAN with perceptual loss outperformed the U-net in terms of errors and similarities between sCT and dCT, and dose estimation for treatment planning of thorax, and abdomen. The sCT generated using CycleGAN produced virtually identical dose distribution maps and dose-volume histograms compared to dCT. CycleGAN with perceptual loss outperformed U-net in sCT generation when trained with weakly paired dCT-MRI for MRgRT. The proposed method will be useful to increase the treatment accuracy of MR-only or MR-guided adaptive radiotherapy.

Supplementary Information

The online version contains supplementary material available at 10.1007/s13534-021-00195-8.

Artificial Intelligence and Radiomics in Head and Neck Cancer Care: Opportunities, Mechanics, and Challenges

The advent of large-scale high-performance computing has allowed the development of machine-learning techniques in oncologic applications. Among these, there has been substantial growth in radiomics (machine-learning texture analysis of images) and artificial intelligence (which uses deep-learning techniques for "learning algorithms"); however, clinical implementation has yet to be realized at scale. To improve implementation, opportunities, mechanics, and challenges, models of imaging-enabled artificial intelligence approaches need to be understood by clinicians who make the treatment decisions. This article aims to convey the basic conceptual premises of radiomics and artificial intelligence using head and neck cancer as a use case. This educational overview focuses on approaches for head and neck oncology imaging, detailing current research efforts and challenges to implementation.

Evaluation of a commercial synthetic computed tomography generation solution for magnetic resonance imaging-only radiotherapy

Purpose

To evaluate the Siemens solution generating Synthetic computed tomography (sCT) for magnetic resonance imaging (MRI)‐only radiotherapy (RT).

Method

A retrospective study was conducted on 47 patients treated with external beam RT for brain or prostate cancer who underwent both MRI and CT for treatment planning. sCT images were generated from MRI using automatic bulk densities segmentation. The geometric accuracy of the sCT was assessed by comparing the Hounsfield Units (HU) difference between sCT and CT for bone structures, soft‐tissue, and full body contour. VMAT plans were computed on the CT for treatment preparation and then copied and recalculated with the same monitor units on the sCT using the AcurosXB algorithm. A 1%‐1mm gamma analysis was performed and DVH metrics for the Planning Target Volume (PTV) like the D_mean and the D_98% were compared. In addition, we evaluate the usability of sCT for daily position verification with cone beam computed tomography (CBCT) for 14 prostate patients by comparing sCT/CBCT registration results to CT/CBCT.

Results

Mean HU differences were small except for the skull (207 HU) and right femoral head of four patients where significant aberrations were found. The mean gamma pass rate was 73.2% for the brain and 84.7% for the prostate and D_mean were smaller than 0.5%. Large differences for the D_98% of the prostate group could be correlated to low Dice index of the PTV. The mean difference of translations and rotations were inferior to 3.5 mm and 0.2° in all directions with a major difference in the anterior‐posterior direction.

Conclusion

The performances of the software were shown to be similar to other sCT generation algorithms in terms of HU difference, dose comparison and daily image localization.

Synthetic CT for single-fraction neoadjuvant partial breast irradiation on an MRI-linac

A synthetic computed tomography (sCT) is required for daily plan optimization on an MRI-linac. Yet, only limited information is available on the accuracy of dose calculations on sCT for breast radiotherapy. This work aimed to (1) evaluate dosimetric accuracy of treatment plans for single-fraction neoadjuvant partial breast irradiation (PBI) on a 1.5 T MRI-linac calculated on a) bulk-density sCT mimicking the current MRI-linac workflow and b) deep learning-generated sCT, and (2) investigate the number of bulk-density levels required. For ten breast cancer patients we created three bulk-density sCTs of increasing complexity from the planning-CT, using bulk-density for: (1) body, lungs, and GTV (sCT_BD1); (2) volumes for sCT_BD1plus chest wall and ipsilateral breast (sCT_BD2); (3) volumes for sCT_BD2plus ribs (sCT_BD3); and a deep learning-generated sCT (sCT_DL) from a 1.5 T MRI in supine position. Single-fraction neoadjuvant PBI treatment plans for a 1.5 T MRI-linac were optimized on each sCT and recalculated on the planning-CT. Image evaluation was performed by assessing mean absolute error (MAE) and mean error (ME) in Hounsfield Units (HU) between the sCTs and the planning-CT. Dosimetric evaluation was performed by assessing dose differences, gamma pass rates, and dose-volume histogram (DVH) differences. The following results were obtained (median across patients for sCT_BD1/sCT_BD2/sCT_BD3/sCT_DLrespectively): MAE inside the body contour was 106/104/104/75 HU and ME was 8/9/6/28 HU, mean dose difference in the PTV_GTVwas 0.15/0.00/0.00/-0.07 Gy, median gamma pass rate (2%/2 mm, 10% dose threshold) was 98.9/98.9/98.7/99.4%, and differences in DVH parameters were well below 2% for all structures except for the skin in the sCT_DL. Accurate dose calculations for single-fraction neoadjuvant PBI on an MRI-linac could be performed on both bulk-density and deep learning sCT, facilitating further implementation of MRI-guided radiotherapy for breast cancer. Balancing simplicity and accuracy, sCT_BD2showed the optimal number of bulk-density levels for a bulk-density approach.

Comparison of Supervised and Unsupervised Deep Learning Methods for Medical Image Synthesis between Computed Tomography and Magnetic Resonance Images

Cross-modality medical image synthesis between magnetic resonance (MR) images and computed tomography (CT) images has attracted increasing attention in many medical imaging area. Many deep learning methods have been used to generate pseudo-MR/CT images from counterpart modality images. In this study, we used U-Net and Cycle-Consistent Adversarial Networks (CycleGAN), which were typical networks of supervised and unsupervised deep learning methods, respectively, to transform MR/CT images to their counterpart modality. Experimental results show that synthetic images predicted by the proposed U-Net method got lower mean absolute error (MAE), higher structural similarity index (SSIM), and peak signal-to-noise ratio (PSNR) in both directions of CT/MR synthesis, especially in synthetic CT image generation. Though synthetic images by the U-Net method has less contrast information than those by the CycleGAN method, the pixel value profile tendency of the synthetic images by the U-Net method is closer to the ground truth images. This work demonstrated that supervised deep learning method outperforms unsupervised deep learning method in accuracy for medical tasks of MR/CT synthesis.

Clinical validation of a commercially available deep learning software for synthetic CT generation for brain

BACKGROUND

Most studies on synthetic computed tomography (sCT) generation for brain rely on in-house developed methods. They often focus on performance rather than clinical feasibility. Therefore, the aim of this work was to validate sCT images generated using a commercially available software, based on a convolutional neural network (CNN) algorithm, to enable MRI-only treatment planning for the brain in a clinical setting.

METHODS

This prospective study included 20 patients with brain malignancies of which 14 had areas of resected skull bone due to surgery. A Dixon magnetic resonance (MR) acquisition sequence for sCT generation was added to the clinical brain MR-protocol. The corresponding sCT images were provided by the software MRI Planner (Spectronic Medical AB, Sweden). sCT images were rigidly registered and resampled to CT for each patient. Treatment plans were optimized on CT and recalculated on sCT images for evaluation of dosimetric and geometric endpoints. Further analysis was also performed for the post-surgical cases. Clinical robustness in patient setup verification was assessed by rigidly registering cone beam CT (CBCT) to sCT and CT images, respectively.

RESULTS

All sCT images were successfully generated. Areas of bone resection due to surgery were accurately depicted. Mean absolute error of the sCT images within the body contour for all patients was 62.2 ± 4.1 HU. Average absorbed dose differences were below 0.2% for parameters evaluated for both targets and organs at risk. Mean pass rate of global gamma (1%/1 mm) for all patients was 100.0 ± 0.0% within PTV and 99.1 ± 0.6% for the full dose distribution. No clinically relevant deviations were found in the CBCT-sCT vs CBCT-CT image registrations. In addition, mean values of voxel-wise patient specific geometric distortion in the Dixon images for sCT generation were below 0.1 mm for soft tissue, and below 0.2 mm for air and bone.

CONCLUSIONS

This work successfully validated a commercially available CNN-based software for sCT generation. Results were comparable for sCT and CT images in both dosimetric and geometric evaluation, for both patients with and without anatomical anomalies. Thus, MRI Planner is feasible to use for radiotherapy treatment planning of brain tumours.

Technical Challenges of Real-Time Adaptive MR-Guided Radiotherapy

In the past few years, radiotherapy (RT) has experienced a major technological innovation with the development of hybrid machines combining magnetic resonance (MR) imaging and linear accelerators. This new technology for MR-guided cancer treatment has the potential to revolutionize the field of adaptive RT due to the opportunity to provide high-resolution, real-time MR imaging before and during treatment application. However, from a technical point of view, several challenges remain which need to be tackled to ensure safe and robust real-time adaptive MR-guided RT delivery. In this manuscript, several technical challenges to MR-guided RT are discussed. Starting with magnetic field strength tradeoffs, the potential and limitations for purely MR-based RT workflows are discussed. Furthermore, the current status of real-time 3D MR imaging and its potential for real-time RT are summarized. Finally, the potential of quantitative MR imaging for future biological RT adaptation is highlighted.

Dosimetric Validation of a GAN-Based Pseudo-CT Generation for MRI-Only Stereotactic Brain Radiotherapy

PURPOSE

Stereotactic radiotherapy (SRT) has become widely accepted as a treatment of choice for patients with a small number of brain metastases that are of an acceptable size, allowing for better target dose conformity, resulting in high local control rates and better sparing of organs at risk. An MRI-only workflow could reduce the risk of misalignment between magnetic resonance imaging (MRI) brain studies and computed tomography (CT) scanning for SRT planning, while shortening delays in planning. Given the absence of a calibrated electronic density in MRI, we aimed to assess the equivalence of synthetic CTs generated by a generative adversarial network (GAN) for planning in the brain SRT setting.

METHODS

All patients with available MRIs and treated with intra-cranial SRT for brain metastases from 2014 to 2018 in our institution were included. After co-registration between the diagnostic MRI and the planning CT, a synthetic CT was generated using a 2D-GAN (2D U-Net). Using the initial treatment plan (Pinnacle v9.10, Philips Healthcare), dosimetric comparison was performed using main dose-volume histogram (DVH) endpoints in respect to ICRU 91 guidelines (Dmax, Dmean, D2%, D50%, D98%) as well as local and global gamma analysis with 1%/1 mm, 2%/1 mm and 2%/2 mm criteria and a 10% threshold to the maximum dose. t-test analysis was used for comparison between the two cohorts (initial and synthetic dose maps).

RESULTS

184 patients were included, with 290 treated brain metastases. The mean number of treated lesions per patient was 1 (range 1-6) and the median planning target volume (PTV) was 6.44 cc (range 0.12-45.41). Local and global gamma passing rates (2%/2 mm) were 99.1 CI95% (98.1-99.4) and 99.7 CI95% (99.6-99.7) respectively (CI: confidence interval). DVHs were comparable, with no significant statistical differences regarding ICRU 91's endpoints.

CONCLUSIONS

Our study is the first to compare GAN-generated CT scans from diagnostic brain MRIs with initial CT scans for the planning of brain stereotactic radiotherapy. We found high similarity between the planning CT and the synthetic CT for both the organs at risk and the target volumes. Prospective validation is under investigation at our institution.

Dosimetric evaluation of synthetic CT image generated using a neural network for MR-only brain radiotherapy

PURPOSE AND BACKGROUND

The magnetic resonance (MR)-only radiotherapy workflow is urged by the increasing use of MR image for the identification and delineation of tumors, while a fast generation of synthetic computer tomography (sCT) image from MR image for dose calculation remains one of the key challenges to the workflow. This study aimed to develop a neural network to generate the sCT in brain site and evaluate the dosimetry accuracy.

MATERIALS AND METHODS

A generative adversarial network (GAN) was developed to translate T1-weighted MRI to sCT. First, the "U-net" shaped encoder-decoder network with some image translation-specific modifications was trained to generate sCT, then the discriminator network was adversarially trained to distinguish between synthetic and real CT images. We enrolled 37 brain cancer patients acquiring both CT and MRI for treatment position simulation. Twenty-seven pairs of 2D T1-weighted MR images and rigidly registered CT image were used to train the GAN model, and the remaining 10 pairs were used to evaluate the model performance through the metric of mean absolute error. Furthermore, the clinical Volume Modulated Arc Therapy plan was calculated on both sCT and real CT, followed by gamma analysis and comparison of dose-volume histogram.

RESULTS

On average, only 15 s were needed to generate one sCT from one T1-weighted MRI. The mean absolute error between synthetic and real CT was 60.52 ± 13.32 Housefield Unit over 5-fold cross validation. For dose distribution on sCT and CT, the average pass rates of gamma analysis using the 3%/3 mm and 2%/2 mm criteria were 99.76% and 97.25% over testing patients, respectively. For parameters of dose-volume histogram for both target and organs at risk, no significant differences were found between both plans.

CONCLUSION

The GAN model can generate synthetic CT from one single MRI sequence within seconds, and a state-of-art accuracy of CT number and dosimetry was achieved.

An MR-only acquisition and artificial intelligence based image-processing protocol for photon and proton therapy using a low field MR

OBJECTIVE

Recent developments on synthetically generated CTs (sCT), hybrid MRI linacs and MR-only simulations underlined the clinical feasibility and acceptance of MR guided radiation therapy. However, considering clinical application of open and low field MR with a limited field of view can result in truncation of the patient's anatomy which further affects the MR to sCT conversion. In this study an acquisition protocol and subsequent MR image stitching is proposed to overcome the limited field of view restriction of open MR scanners, for MR-only photon and proton therapy.

MATERIAL AND METHODS

12 prostate cancer patients scanned with an open 0.35T scanner were included. To obtain the full body contour an enhanced imaging protocol including two repeated scans after bilateral table movement was introduced. All required structures (patient contour, target and organ at risk) were delineated on a post-processed combined transversal image set (stitched MRI). The postprocessed MR was converted into a sCT by a pretrained neural network generator. Inversely planned photon and proton plans (VMAT and SFUD) were designed using the sCT and recalculated for rigidly and deformably registered CT images and compared based on D2%, D50%, V70Gy for organs at risk and based on D2%, D50%, D98% for the CTV and PTV. The stitched MRI and the untruncated MRI were compared to the CT, and the maximum surface distance was calculated. The sCT was evaluated with respect to delineation accuracy by comparing on stitched MRI and sCT using the DICE coefficient for femoral bones and the whole body.

RESULTS

Maximum surface distance analysis revealed uncertainties in lateral direction of 1-3mm on average. DICE coefficient analysis confirms good performance of the sCT conversion, i.e. 92%, 93%, and 100% were obtained for femoral bone left and right and whole body. Dose comparison resulted in uncertainties below 1% between deformed CT and sCT and below 2% between rigidly registered CT and sCT in the CTV for photon and proton treatment plans.

DISCUSSION

A newly developed acquisition protocol for open MR scanners and subsequent Sct generation revealed good acceptance for photon and proton therapy. Moreover, this protocol tackles the restriction of the limited FOVs and expands the capacities towards MR guided proton therapy with horizontal beam lines.

Synthetic computed tomography data allows for accurate absorbed dose calculations in a magnetic resonance imaging only workflow for head and neck radiotherapy

•

The geometry of the synthetic CT is comparable to the CT in the H&N region.

•

Synthetic CT in the H&N region provides similar absorbed dose calculation as the CT.

•

Absorbed dose calculations in the dental region could benefit from using synthetic CT.

Background and purpose

Few studies on magnetic resonance imaging (MRI) only head and neck radiation treatment planning exist, and none using a generally available software. The aim of this study was to evaluate the accuracy of absorbed dose for head and neck synthetic computed tomography data (sCT) generated by a commercial convolutional neural network-based algorithm.

Materials and methods

For 44 head and neck cancer patients, sCT were generated and the geometry was validated against computed tomography data (CT). The clinical CT based treatment plan was transferred to the sCT and recalculated without re-optimization, and differences in relative absorbed dose were determined for dose-volume-histogram (DVH) parameters and the 3D volume.

Results

For overall body, the results of the geometric validation were (Mean ± 1sd): Mean error −5 ± 10 HU, mean absolute error 67 ± 14 HU, Dice similarity coefficient 0.98 ± 0.05, and Hausdorff distance difference 4.2 ± 1.7 mm. Water equivalent depth difference for region Th1-C7, mid mandible and mid nose were −0.3 ± 3.4, 1.1 ± 2.0 and 0.7 ± 3.8 mm respectively. The maximum mean deviation in absorbed dose for all DVH parameters was 0.30% (0.12 Gy). The absorbed doses were considered equivalent (p-value < 0.001) and the mean 3D gamma passing rate was 99.4 (range: 95.7–99.9%).

Conclusions

The convolutional neural network-based algorithm generates sCT which allows for accurate absorbed dose calculations for MRI-only head and neck radiation treatment planning. The sCT allows for statistically equivalent absorbed dose calculations compared to CT based radiotherapy.

A review on medical imaging synthesis using deep learning and its clinical applications

This paper reviewed the deep learning-based studies for medical imaging synthesis and its clinical application. Specifically, we summarized the recent developments of deep learning-based methods in inter- and intra-modality image synthesis by listing and highlighting the proposed methods, study designs, and reported performances with related clinical applications on representative studies. The challenges among the reviewed studies were then summarized with discussion.

Glide‐Hurst C. Performance of deep learning synthetic CTs for MR-only brain radiation therapy

PURPOSE

To evaluate the dosimetric and image-guided radiation therapy (IGRT) performance of a novel generative adversarial network (GAN) generated synthetic CT (synCT) in the brain and compare its performance for clinical use including conventional brain radiotherapy, cranial stereotactic radiosurgery (SRS), planar, and volumetric IGRT.

METHODS AND MATERIALS

SynCT images for 12 brain cancer patients (6 SRS, 6 conventional) were generated from T1-weighted postgadolinium magnetic resonance (MR) images by applying a GAN model with a residual network (ResNet) generator and a convolutional neural network (CNN) with 5 convolutional layers as the discriminator that classified input images as real or synthetic. Following rigid registration, clinical structures and treatment plans derived from simulation CT (simCT) images were transferred to synCTs. Dose was recalculated for 15 simCT/synCT plan pairs using fixed monitor units. Two-dimensional (2D) gamma analysis (2%/2 mm, 1%/1 mm) was performed to compare dose distributions at isocenter. Dose-volume histogram (DVH) metrics (D95% , D99% , D0.2cc, and D0.035cc ) were assessed for the targets and organ at risks (OARs). IGRT performance was evaluated via volumetric registration between cone beam CT (CBCT) to synCT/simCT and planar registration between KV images to synCT/simCT digital reconstructed radiographs (DRRs).

RESULTS

Average gamma passing rates at 1%/1mm and 2%/2mm were 99.0 ± 1.5% and 99.9 ± 0.2%, respectively. Excellent agreement in DVH metrics was observed (mean difference ≤0.10 ± 0.04 Gy for targets, 0.13 ± 0.04 Gy for OARs). The population averaged mean difference in CBCT-synCT registrations were <0.2 mm and 0.1 degree different from simCT-based registrations. The mean difference between kV-synCT DRR and kV-simCT DRR registrations was <0.5 mm with no statistically significant differences observed (P > 0.05). An outlier with a large resection cavity exhibited the worst-case scenario.

CONCLUSION

Brain GAN synCTs demonstrated excellent performance for dosimetric and IGRT endpoints, offering potential use in high precision brain cancer therapy.