Deep Convolution Neural Network (DCNN) Multiplane Approach to Synthetic CT Generation From MR images—Application in Brain Proton Therapy

Dosimetric impact of geometric distortions in an MRI-only proton therapy workflow for lung, liver and pancreas

In a radiation therapy workflow based on Magnetic Resonance Imaging (MRI), dosimetric errors may arise due to geometric distortions introduced by MRI. The aim of this study was to quantify the dosimetric effect of system-dependent geometric distortions in an MRI-only workflow for proton therapy applied at extra-cranial sites. An approach was developed, in which computed tomography (CT) images were distorted using an MRI displacement map, which represented the MR distortions in a spoiled gradient-echo sequence due to gradient nonlinearities and static magnetic field inhomogeneities. A retrospective study was conducted on 4DCT/MRI digital phantoms and 18 4DCT clinical datasets of the thoraco-abdominal site. The treatment plans were designed and separately optimized for each beam in a beam specific Planning Target Volume on the distorted CT, and the final dose distribution was obtained as the average. The dose was then recalculated in undistorted CT using the same beam geometry and beam weights. The analysis was performed in terms of Dose Volume Histogram (DVH) parameters. No clinically relevant dosimetric impact was observed on organs at risk, whereas in the target structure, geometric distortions caused statistically significant variations in the planned dose DVH parameters and dose homogeneity index (DHI). The dosimetric variations in the target structure were smaller in abdominal cases (ΔD_2%, ΔD_98%, and ΔD_mean all below 0.1% and ΔDHI below 0.003) compared to the lung cases. Indeed, lung patients with tumors isolated inside lung parenchyma exhibited higher dosimetric variations (ΔD_2%≥0.3%, ΔD_98%≥15.9%, ΔD_mean≥3.3% and ΔDHI≥0.102) than lung patients with tumor close to soft tissue (ΔD_2%≤0.4%, ΔD_98%≤5.6%, ΔD_mean≤0.9% and ΔDHI≤0.027) potentially due to higher density variations along the beam path. Results suggest the potential applicability of MRI-only proton therapy, provided that specific analysis is applied for isolated lung tumors.

Self-derived organ attention for unpaired CT-MRI deep domain adaptation based MRI segmentation

To develop and evaluate a deep learning method to segment parotid glands from MRI using unannotated MRI and unpaired expert-segmented CT datasets. We introduced a new self-derived organ attention deep learning network for combined CT to MRI image-to-image translation (I2I) and MRI segmentation, all trained as an end-to-end network. The expert segmentations available on CT scans were combined with the I2I translated pseudo MR images to train the MRI segmentation network. Once trained, the MRI segmentation network alone is required. We introduced an organ attention discriminator that constrains the CT to MR generator to synthesize pseudo MR images that preserve organ geometry and appearance statistics as in real MRI. The I2I translation network training was regularized using the organ attention discriminator, global image-matching discriminator, and cycle consistency losses. MRI segmentation training was regularized by using cross-entropy loss. Segmentation performance was compared against multiple domain adaptation-based segmentation methods using the Dice similarity coefficient (DSC) and Hausdorff distance at the 95th percentile (HD95). All networks were trained using 85 unlabeled T2-weighted fat suppressed (T2wFS) MRIs and 96 expert-segmented CT scans. Performance upper-limit was based on fully supervised MRI training done using the 85 T2wFS MRI with expert segmentations. Independent evaluation was performed on 77 MRIs never used in training. The proposed approach achieved the highest accuracy (left parotid: DSC 0.82 ± 0.03, HD95 2.98 ± 1.01 mm; right parotid: 0.81 ± 0.05, HD95 3.14 ± 1.17 mm) compared to other methods. This accuracy was close to the reference fully supervised MRI segmentation (DSC of 0.84 ± 0.04, a HD95 of 2.24 ± 0.77 mm for the left parotid, and a DSC of 0.84 ± 0.06 and HD95 of 2.32 ± 1.37 mm for the right parotid glands).

Overview of artificial intelligence-based applications in radiotherapy: Recommendations for implementation and quality assurance

Artificial Intelligence (AI) is currently being introduced into different domains, including medicine. Specifically in radiation oncology, machine learning models allow automation and optimization of the workflow. A lack of knowledge and interpretation of these AI models can hold back wide-spread and full deployment into clinical practice. To facilitate the integration of AI models in the radiotherapy workflow, generally applicable recommendations on implementation and quality assurance (QA) of AI models are presented. For commonly used applications in radiotherapy such as auto-segmentation, automated treatment planning and synthetic computed tomography (sCT) the basic concepts are discussed in depth. Emphasis is put on the commissioning, implementation and case-specific and routine QA of AI models needed for a methodical introduction in clinical practice.

Data reduction and data visualization for automatic diagnosis using gene expression and clinical data

Accurate diagnoses of specific diseases require, in general, the review of the whole medical history of a patient. Currently, even though many advances have been made for disease monitoring, domain experts are still requested to perform direct analyses in order to get a precise classification, thus implying significant efforts and costs. In this work we present a framework for automated diagnosis based on high-dimensional gene expression and clinical data. Given that high-dimensional data can be difficult to analyze and computationally expensive to process, we first perform data reduction to transform high-dimensional representations of data into a lower dimensional space, yet keeping them meaningful for our purposes. We used then different data visualization techniques to embed complex pieces of information in 2-D images, that are in turn used to perform diagnosis relying on deep learning approaches. Experimental analyses show that the proposed method achieves good performance, featuring a prediction Recall value between 91% and 99%.

A Review on Advances in Intra-operative Imaging for Surgery and Therapy: Imagining the Operating Room of the Future

With the advent of Minimally Invasive Surgery (MIS), intra-operative imaging has become crucial for surgery and therapy guidance, allowing to partially compensate for the lack of information typical of MIS. This paper reviews the advancements in both classical (i.e. ultrasounds, X-ray, optical coherence tomography and magnetic resonance imaging) and more recent (i.e. multispectral, photoacoustic and Raman imaging) intra-operative imaging modalities. Each imaging modality was analyzed, focusing on benefits and disadvantages in terms of compatibility with the operating room, costs, acquisition time and image characteristics. Tables are included to summarize this information. New generation of hybrid surgical room and algorithms for real time/in room image processing were also investigated. Each imaging modality has its own (site- and procedure-specific) peculiarities in terms of spatial and temporal resolution, field of view and contrasted tissues. Besides the benefits that each technique offers for guidance, considerations about operators and patient risk, costs, and extra time required for surgical procedures have to be considered. The current trend is to equip surgical rooms with multimodal imaging systems, so as to integrate multiple information for real-time data extraction and computer-assisted processing. The future of surgery is to enhance surgeons eye to minimize intra- and after-surgery adverse events and provide surgeons with all possible support to objectify and optimize the care-delivery process.

MR-guided proton therapy: a review and a preview

Background

The targeting accuracy of proton therapy (PT) for moving soft-tissue tumours is expected to greatly improve by real-time magnetic resonance imaging (MRI) guidance. The integration of MRI and PT at the treatment isocenter would offer the opportunity of combining the unparalleled soft-tissue contrast and real-time imaging capabilities of MRI with the most conformal dose distribution and best dose steering capability provided by modern PT. However, hybrid systems for MR-integrated PT (MRiPT) have not been realized so far due to a number of hitherto open technological challenges. In recent years, various research groups have started addressing these challenges and exploring the technical feasibility and clinical potential of MRiPT. The aim of this contribution is to review the different aspects of MRiPT, to report on the status quo and to identify important future research topics.

Methods

Four aspects currently under study and their future directions are discussed: modelling and experimental investigations of electromagnetic interactions between the MRI and PT systems, integration of MRiPT workflows in clinical facilities, proton dose calculation algorithms in magnetic fields, and MRI-only based proton treatment planning approaches.

Conclusions

Although MRiPT is still in its infancy, significant progress on all four aspects has been made, showing promising results that justify further efforts for research and development to be undertaken. First non-clinical research solutions have recently been realized and are being thoroughly characterized. The prospect that first prototype MRiPT systems for clinical use will likely exist within the next 5 to 10 years seems realistic, but requires significant work to be performed by collaborative efforts of research groups and industrial partners.

Investigating conditional GAN performance with different generator architectures, an ensemble model, and different MR scanners for MR-sCT conversion

Recent developments in magnetic resonance (MR) to synthetic computed tomography (sCT) conversion have shown that treatment planning is possible without an initial planning CT. Promising conversion results have been demonstrated recently using conditional generative adversarial networks (cGANs). However, the performance is generally only tested on images from one MR scanner, which neglects the potential of neural networks to find general high-level abstract features. In this study, we explored the generalizability of the generator models, trained on a single field strength scanner, to data acquired with higher field strengths. T2-weighted 0.35T MRIs and CTs from 51 patients treated for prostate (40) and cervical cancer (11) were included. 25 of them were used to train four different generators (SE-ResNet, DenseNet, U-Net, and Embedded Net). Further, an ensemble model was created from the four network outputs. The models were validated on 16 patients from a 0.35T MR scanner. Further, the trained models were tested on the Gold Atlas dataset, containing T2-weighted MR scans of different field strengths; 1.5T(7) and 3T(12), and 10 patients from the 0.35T scanner. The sCTs were dosimetrically compared using clinical VMAT plans for all test patients. For the same scanner (0.35T), the results from the different models were comparable on the test set, with only minor differences in the mean absolute error (MAE) (35-51HU body). Similar results were obtained for conversions of 3T GE Signa and the 3T GE Discovery images (40-62HU MAE) for three of the models. However, larger differences were observed for the 1.5T images (48-65HU MAE). The overall best model was found to be the ensemble model. All dose differences were below 1%. This study shows that it is possible to generalize models trained on images of one scanner to other scanners and different field strengths. The best metric results were achieved by the combination of all networks.

Comparison of CBCT based synthetic CT methods suitable for proton dose calculations in adaptive proton therapy

In-room imaging is a prerequisite for adaptive proton therapy. The use of onboard cone-beam computed tomography (CBCT) imaging, which is routinely acquired for patient position verification, can enable daily dose reconstructions and plan adaptation decisions. Image quality deficiencies though, hamper dose calculation accuracy and make corrections of CBCTs a necessity. This study compared three methods to correct CBCTs and create synthetic CTs that are suitable for proton dose calculations. CBCTs, planning CTs and repeated CTs (rCT) from 33 H&N cancer patients were used to compare a deep convolutional neural network (DCNN), deformable image registration (DIR) and an analytical image-based correction method (AIC) for synthetic CT (sCT) generation. Image quality of sCTs was evaluated by comparison with a same-day rCT, using mean absolute error (MAE), mean error (ME), Dice similarity coefficient (DSC), structural non-uniformity (SNU) and signal/contrast-to-noise ratios (SNR/CNR) as metrics. Dosimetric accuracy was investigated in an intracranial setting by performing gamma analysis and calculating range shifts. Neural network-based sCTs resulted in the lowest MAE and ME (37/2 HU) and the highest DSC (0.96). While DIR and AIC generated images with a MAE of 44/77 HU, a ME of -8/1 HU and a DSC of 0.94/0.90. Gamma and range shift analysis showed almost no dosimetric difference between DCNN and DIR based sCTs. The lower image quality of AIC based sCTs affected dosimetric accuracy and resulted in lower pass ratios and higher range shifts. Patient-specific differences highlighted the advantages and disadvantages of each method. For the set of patients, the DCNN created synthetic CTs with the highest image quality. Accurate proton dose calculations were achieved by both DCNN and DIR based sCTs. The AIC method resulted in lower image quality and dose calculation accuracy was reduced compared to the other methods.

Dosimetric evaluation of synthetic CT generated with GANs for MRI-only proton therapy treatment planning of brain tumors

PURPOSE

The purpose of this study was to address the dosimetric accuracy of synthetic computed tomography (sCT) images of patients with brain tumor generated using a modified generative adversarial network (GAN) method, for their use in magnetic resonance imaging (MRI)-only treatment planning for proton therapy.

METHODS

Dose volume histogram (DVH) analysis was performed on CT and sCT images of patients with brain tumor for plans generated for intensity-modulated proton therapy (IMPT). All plans were robustly optimized using a commercially available treatment planning system (RayStation, from RaySearch Laboratories) and standard robust parameters reported in the literature. The IMPT plan was then used to compute the dose on CT and sCT images for dosimetric comparison, using RayStation analytical (pencil beam) dose algorithm. We used a second, independent Monte Carlo dose calculation engine to recompute the dose on both CT and sCT images to ensure a proper analysis of the dosimetric accuracy of the sCT images.

RESULTS

The results extracted from RayStation showed excellent agreement for most DVH metrics computed on the CT and sCT for the nominal case, with a mean absolute difference below 0.5% (0.3 Gy) of the prescription dose for the clinical target volume (CTV) and below 2% (1.2 Gy) for the organs at risk (OARs) considered. This demonstrates a high dosimetric accuracy for the generated sCT images, especially in the target volume. The metrics obtained from the Monte Carlo doses mostly agreed with the values extracted from RayStation for the nominal and worst-case scenarios (mean difference below 3%).

CONCLUSIONS

This work demonstrated the feasibility of using sCT generated with a GAN-based deep learning method for MRI-only treatment planning of patients with brain tumor in intensity-modulated proton therapy.

Deep Convolutional Neural Networks Based Analysis of Cephalometric Radiographs for Differential Diagnosis of Orthognathic Surgery Indications

The aim of this study was to evaluate the deep convolutional neural networks (DCNNs) based on analysis of cephalometric radiographs for the differential diagnosis of the indications of orthognathic surgery. Among the DCNNs, Modified-Alexnet, MobileNet, and Resnet50 were used, and the accuracy of the models was evaluated by performing 4-fold cross validation. Additionally, gradient-weighted class activation mapping (Grad-CAM) was used to perform visualized interpretation to determine which region affected the DCNNs’ class classification. The prediction accuracy of the models was 96.4% for Modified-Alexnet, 95.4% for MobileNet, and 95.6% for Resnet50. According to the Grad-CAM analysis, the most influential regions for the DCNNs’ class classification were the maxillary and mandibular teeth, mandible, and mandibular symphysis. This study suggests that DCNNs-based analysis of cephalometric radiograph images can be successfully applied for differential diagnosis of the indications of orthognathic surgery.

MRI-based IMPT planning for prostate cancer

PURPOSE

Treatment planning for proton therapy requires the relative proton stopping power ratio (RSP) information of the patient for accurate dose calculations. RSP are conventionally obtained after mapping of the Hounsfield units (HU) from a calibrated patient computed tomography (CT). One or multiple CT are needed for a given treatment which represents additional, undesired dose to the patient. For prostate cancer, magnetic resonance imaging (MRI) scans are the gold standard for segmentation while offering dose-less imaging. We here quantify the clinical applicability of converted MR images as a substitute for intensity modulated proton therapy (IMPT) treatment of the prostate.

METHODS

MRCAT (Magnetic Resonance for Calculating ATtenuation) is a Philips-developed technology which produces a synthetic CT image consisting of five HU from a specific set of MRI acquisitions. MRCAT and original planning CT data sets were obtained for ten patients. An IMPT plan was generated on the MRCAT for each patient. Plans were produced such that they fulfill the prostate protocol in use at Massachusetts General Hospital (MGH). The plans were then recomputed onto the nominal planning CT for each patient. Robustness analyses (±5 mm setup shifts and ±3.5 % range uncertainties) were also performed.

RESULTS

Comparison of MRCAT plans and their recomputation onto the planning CT plan showed excellent agreement. Likewise, dose perturbations due to setup shifts and range uncertainties were well within clinical acceptance demonstrating the clinical viability of the approach.

CONCLUSIONS

This work demonstrate the clinical acceptability of substituting MR converted RSP images instead of CT for IMPT planning of prostate cancer. This further translates into higher contouring accuracy along with lesser imaging dose.