DoseGAN: a generative adversarial network for synthetic dose prediction using attention-gated discrimination and generation

Transformer-Integrated Hybrid Convolutional Neural Network for Dose Prediction in Nasopharyngeal Carcinoma Radiotherapy

Radiotherapy is recognized as the major treatment of nasopharyngeal carcinoma. Rapid and accurate dose prediction can improve the efficiency of the treatment planning process and the quality of radiotherapy plans. Currently, deep learning-based methods have been widely applied to dose prediction for radiotherapy treatment planning. However, it is important to note that existing models based on Convolutional Neural Networks (CNN) often overlook long-distance information. Although some studies try to use Transformer to solve the problem, it lacks the ability of CNN to process the spatial information inherent in images. Therefore, we propose a novel CNN and Transformer hybrid dose prediction model. To enhance the transmission ability of features between CNN and Transformer, we design a hierarchical dense recurrent encoder with a channel attention mechanism. Additionally, we propose a progressive decoder that preserves richer texture information through layer-wise reconstruction of high-dimensional feature maps. The proposed model also introduces object-driven skip connections, which facilitate the flow of information between the encoder and decoder. Experiments are conducted on in-house datasets, and the results show that the proposed model is superior to baseline methods in most dosimetric criteria. In addition, the image analysis metrics including PSNR, SSIM, and NRMSE demonstrate that the proposed model is consistent with ground truth and produces promising visual effects compared to other advanced methods. The proposed method could be taken as a powerful clinical guidance tool for physicists, significantly enhancing the efficiency of radiotherapy planning. The source code is available at https://github.com/CDUTJ102/THCN-Net .

FA-Net: A hierarchical feature fusion and interactive attention-based network for dose prediction in liver cancer patients

Dose prediction is a crucial step in automated radiotherapy planning for liver cancer. Several deep learning-based approaches for dose prediction have been proposed to enhance the design efficiency and quality of radiotherapy plan. However, these approaches usually take CT images and contours of organs at risk (OARs) and planning target volume (PTV) as a multi-channel input and is thus difficult to extract sufficient feature information from each input, which results in unsatisfactory dose distribution. In this paper, we propose a novel dose prediction network for liver cancer based on hierarchical feature fusion and interactive attention. A feature extraction module is first constructed to extract multi-scale features from different inputs, and a hierarchical feature fusion module is then built to fuse these multi-scale features hierarchically. A decoder based on attention mechanism is designed to gradually reconstruct the fused features into dose distribution. Additionally, we design an autoencoder network to generate a perceptual loss during training stage, which is used to improve the accuracy of dose prediction. The proposed method is tested on private clinical dataset and obtains HI and CI of 0.31 and 0.87, respectively. The experimental results are better than those by several existing methods, indicating that the dose distribution generated by the proposed method is close to that approved in clinics. The codes are available at https://github.com/hired-ld/FA-Net.

Predicting the 3-Dimensional Dose Distribution of Multilesion Lung Stereotactic Ablative Radiation Therapy With Generative Adversarial Networks

PURPOSE

Because SABR therapy is being used to treat greater numbers of lung metastases, selecting the optimal dose and fractionation to balance local failure and treatment toxicity becomes increasingly challenging. Multilesion lung SABR therapy plans include spatially diverse lesions with heterogeneous prescriptions and interacting dose distributions. In this study, we developed and evaluated a generative adversarial network (GAN) to provide real-time dosimetry predictions for these complex cases.

METHODS AND MATERIALS

A GAN was trained to predict dosimetry on a data set of patients who received SABR therapy for lung lesions at a tertiary center. Model input included the planning computed tomography scan, the organs at risk (OARs) and target structures, and an initial estimate of exponential dose fall-off. Multilesion plans were split 80/20 for training and evaluation. Models were evaluated on voxel-voxel, clinical dose-volume histogram, and conformality metrics. An out-of-sample validation and analysis of model variance were performed.

RESULTS

There were 125 multilesion plans from 102 patients with 357 lesions. Patients were treated for 2 to 7 lesions, with 19 unique dose-fractionation schemes over 1 to 3 courses of treatment. The out-of-sample validation set contained an additional 90 plans from 80 patients. The mean absolute difference and gamma pass fraction between the predicted and true dosimetry was <3 Gy and >90% for all OARs. The absolute differences in lung V20 and CV14 were 1.40% ± 0.99% and 75.8 ± 42.0 cc, respectively. The ratios of predicted to true R50%, R100%, and D2cm were 1.00 ± 0.16, 0.96 ± 0.32, and 1.01 ± 0.36, respectively. The out-of-sample validation set maintained mean absolute difference and gamma pass fraction of <3 Gy and >90%, respectively for all OARs. The median standard deviation of variance in V20 and CV14 prediction was 0.49% and 22.2 cc, respectively.

CONCLUSIONS

A GAN for predicting the 3-D dosimetry of complex multilesion lung SABR therapy is presented. Rapid dosimetry prediction can be used to assess treatment feasibility and explore dosimetric differences between varying prescriptions.

Deep learning architecture with shunted transformer and 3D deformable convolution for voxel-level dose prediction of head and neck tumors

Intensity-modulated radiation therapy (IMRT) has been widely used in treating head and neck tumors. However, due to the complex anatomical structures in the head and neck region, it is challenging for the plan optimizer to rapidly generate clinically acceptable IMRT treatment plans. A novel deep learning multi-scale Transformer (MST) model was developed in the current study aiming to accelerate the IMRT planning for head and neck tumors while generating more precise prediction of the voxel-level dose distribution. The proposed end-to-end MST model employs the shunted Transformer to capture multi-scale features and learn a global dependency, and utilizes 3D deformable convolution bottleneck blocks to extract shape-aware feature and compensate the loss of spatial information in the patch merging layers. Moreover, data augmentation and self-knowledge distillation are used to further improve the prediction performance of the model. The MST model was trained and evaluated on the OpenKBP Challenge dataset. Its prediction accuracy was compared with three previous dose prediction models: C3D, TrDosePred, and TSNet. The predicted dose distributions of our proposed MST model in the tumor region are closest to the original clinical dose distribution. The MST model achieves the dose score of 2.23 Gy and the DVH score of 1.34 Gy on the test dataset, outperforming the other three models by 8%-17%. For clinical-related DVH dosimetric metrics, the prediction accuracy in terms of mean absolute error (MAE) is 2.04% for D 99 , 1.54% for D 95 , 1.87% for D 1 , 1.87% for D mean , 1.89% for D 0.1 c c , respectively, superior to the other three models. The quantitative results demonstrated that the proposed MST model achieved more accurate voxel-level dose prediction than the previous models for head and neck tumors. The MST model has a great potential to be applied to other disease sites to further improve the quality and efficiency of radiotherapy planning.

Clinical implementation and evaluation of deep learning-assisted automatic radiotherapy treatment planning for lung cancer

PURPOSE

The purpose of the study is to investigate the clinical application of deep learning (DL)-assisted automatic radiotherapy planning for lung cancer.

METHODS

A DL model was developed for predicting patient-specific doses, trained and validated on a dataset of 235 patients with diverse target volumes and prescriptions. The model was integrated into clinical workflow with DL-predicted objective functions. The automatic plans were retrospectively designed for additional 50 treated manual volumetric modulated arc therapy (VMAT) plans. A comparison was made between automatic and manual plans in terms of dosimetric indexes, monitor units (MUs) and planning time. Plan quality metric (PQM) encompassing these indexes was evaluated, with higher PQM values indicating superior plan quality. Qualitative evaluations of two plans were conducted by four reviewers.

RESULTS

The PQM score was 40.7 ± 13.1 for manual plans and 40.8 ± 13.5 for automatic plans (P = 0.75). Compared to manual plans, the targets coverage and homogeneity of automatic plans demonstrated no significant difference. Manual plans exhibited better sparing for lung in V5 (difference: 1.8 ± 4.2 %, P = 0.02), whereas automatic plans showed enhanced sparing for heart in V30 (difference: 1.4 ± 4.7 %, P = 0.02) and for spinal cord in Dmax (difference: 0.7 ± 4.7 Gy, P = 0.04). The planning time and MUs of automatic plans were significantly reduced by 70.5 ± 20.0 min and 97.4 ± 82.1. Automatic plans were deemed acceptable in 88 % of the reviews (176/200).

CONCLUSIONS

The DL-assisted approach for lung cancer notably decreased planning time and MUs, while demonstrating comparable or superior quality relative to manual plans. It has the potential to provide benefit to lung cancer patients.

Deep learning based linear energy transfer calculation for proton therapy

Objective.This study aims to address the limitations of traditional methods for calculating linear energy transfer (LET), a critical component in assessing relative biological effectiveness (RBE). Currently, Monte Carlo (MC) simulation, the gold-standard for accuracy, is resource-intensive and slow for dose optimization, while the speedier analytical approximation has compromised accuracy. Our objective was to prototype a deep-learning-based model for calculating dose-averaged LET (LETd) using patient anatomy and dose-to-water (DW) data, facilitating real-time biological dose evaluation and LET optimization within proton treatment planning systems.Approach. 275 4-field prostate proton Stereotactic Body Radiotherapy plans were analyzed, rendering a total of 1100 fields. Those were randomly split into 880, 110, and 110 fields for training, validation, and testing. A 3D Cascaded UNet model, along with data processing and inference pipelines, was developed to generate patient-specific LETddistributions from CT images and DW. The accuracy of the LETdof the test dataset was evaluated against MC-generated ground truth through voxel-based mean absolute error (MAE) and gamma analysis.Main results.The proposed model accurately inferred LETddistributions for each proton field in the test dataset. A single-field LETdcalculation took around 100 ms with trained models running on a NVidia A100 GPU. The selected model yielded an average MAE of 0.94 ± 0.14 MeV cm-1and a gamma passing rate of 97.4% ± 1.3% when applied to the test dataset, with the largest discrepancy at the edge of fields where the dose gradient was the largest and counting statistics was the lowest.Significance.This study demonstrates that deep-learning-based models can efficiently calculate LETdwith high accuracy as a fast-forward approach. The model shows great potential to be utilized for optimizing the RBE of proton treatment plans. Future efforts will focus on enhancing the model's performance and evaluating its adaptability to different clinical scenarios.

A cascade transformer-based model for 3D dose distribution prediction in head and neck cancer radiotherapy

Objective. Radiation therapy is one of the primary methods used to treat cancer in the clinic. Its goal is to deliver a precise dose to the planning target volume while protecting the surrounding organs at risk (OARs). However, the traditional workflow used by dosimetrists to plan the treatment is time-consuming and subjective, requiring iterative adjustments based on their experience. Deep learning methods can be used to predict dose distribution maps to address these limitations.Approach. The study proposes a cascade model for OARs segmentation and dose distribution prediction. An encoder-decoder network has been developed for the segmentation task, in which the encoder consists of transformer blocks, and the decoder uses multi-scale convolutional blocks. Another cascade encoder-decoder network has been proposed for dose distribution prediction using a pyramid architecture. The proposed model has been evaluated using an in-house head and neck cancer dataset of 96 patients and OpenKBP, a public head and neck cancer dataset of 340 patients.Main results. The segmentation subnet achieved 0.79 and 2.71 for Dice and HD95 scores, respectively. This subnet outperformed the existing baselines. The dose distribution prediction subnet outperformed the winner of the OpenKBP2020 competition with 2.77 and 1.79 for dose and dose-volume histogram scores, respectively. Besides, the end-to-end model, including both subnets simultaneously, outperformed the related studies.Significance. The predicted dose maps showed good coincidence with ground-truth, with a superiority after linking with the auxiliary segmentation task. The proposed model outperformed state-of-the-art methods, especially in regions with low prescribed doses. The codes are available athttps://github.com/GhTara/Dose_Prediction.

Attention-Gated Deep-Learning-Based Automatic Digitization of Interstitial Needles in High-Dose-Rate Brachytherapy for Cervical Cancer

Purpose

Deep learning can be used to automatically digitize interstitial needles in high-dose-rate (HDR) brachytherapy for patients with cervical cancer. The aim of this study was to design a novel attention-gated deep-learning model, which may further improve the accuracy of and better differentiate needles.

Methods and Materials

Seventeen patients with cervical cancer with 56 computed tomography-based interstitial HDR brachytherapy plans from the local hospital were retrospectively chosen with the local institutional review board's approval. Among them, 50 plans were randomly selected as the training set and the rest as the validation set. Spatial and channel attention gates (AGs) were added to 3-dimensional convolutional neural networks (CNNs) to highlight needle features and suppress irrelevant regions; this was supposed to facilitate convergence and improve accuracy of automatic needle digitization. Subsequently, the automatically digitized needles were exported to the Oncentra treatment planning system (Elekta Solutions AB, Stockholm, Sweden) for dose evaluation. The geometric and dosimetric accuracy of automatic needle digitization was compared among 3 methods: (1) clinically approved plans with manual needle digitization (ground truth); (2) the conventional deep-learning (CNN) method; and (3) the attention-added deep-learning (CNN + AG) method, in terms of the Dice similarity coefficient (DSC), tip and shaft positioning errors, dose distribution in the high-risk clinical target volume (HR-CTV), organs at risk, and so on.

Results

The attention-gated CNN model was superior to CNN without AGs, with a greater DSC (approximately 94% for CNN + AG vs 89% for CNN). The needle tip and shaft errors of the CNN + AG method (1.1 mm and 1.8 mm, respectively) were also much smaller than those of the CNN method (2.0 mm and 3.3 mm, respectively). Finally, the dose difference for the HR-CTV D90 using the CNN + AG method was much more accurate than that using CNN (0.4% and 1.7%, respectively).

Conclusions

The attention-added deep-learning model was successfully implemented for automatic needle digitization in HDR brachytherapy, with clinically acceptable geometric and dosimetric accuracy. Compared with conventional deep-learning neural networks, attention-gated deep learning may have superior performance and great clinical potential.

Unsupervised Domain Adaptive Dose Prediction via Cross-Attention Transformer and Target-Specific Knowledge Preservation

Radiotherapy is one of the leading treatments for cancer. To accelerate the implementation of radiotherapy in clinic, various deep learning-based methods have been developed for automatic dose prediction. However, the effectiveness of these methods heavily relies on the availability of a substantial amount of data with labels, i.e. the dose distribution maps, which cost dosimetrists considerable time and effort to acquire. For cancers of low-incidence, such as cervical cancer, it is often a luxury to collect an adequate amount of labeled data to train a well-performing deep learning (DL) model. To mitigate this problem, in this paper, we resort to the unsupervised domain adaptation (UDA) strategy to achieve accurate dose prediction for cervical cancer (target domain) by leveraging the well-labeled high-incidence rectal cancer (source domain). Specifically, we introduce the cross-attention mechanism to learn the domain-invariant features and develop a cross-attention transformer-based encoder to align the two different cancer domains. Meanwhile, to preserve the target-specific knowledge, we employ multiple domain classifiers to enforce the network to extract more discriminative target features. In addition, we employ two independent convolutional neural network (CNN) decoders to compensate for the lack of spatial inductive bias in the pure transformer and generate accurate dose maps for both domains. Furthermore, to enhance the performance, two additional losses, i.e. a knowledge distillation loss (KDL) and a domain classification loss (DCL), are incorporated to transfer the domain-invariant features while preserving domain-specific information. Experimental results on a rectal cancer dataset and a cervical cancer dataset have demonstrated that our method achieves the best quantitative results with [Formula: see text], [Formula: see text], and HI of 1.446, 1.231, and 0.082, respectively, and outperforms other methods in terms of qualitative assessment.

TransDose: Transformer-based radiotherapy dose prediction from CT images guided by super-pixel-level GCN classification

Radiotherapy is a mainstay treatment for cancer in clinic. An excellent radiotherapy treatment plan is always based on a high-quality dose distribution map which is produced by repeated manual trial-and-errors of experienced experts. To accelerate the radiotherapy planning process, many automatic dose distribution prediction methods have been proposed recently and achieved considerable fruits. Nevertheless, these methods require certain auxiliary inputs besides CT images, such as segmentation masks of the tumor and organs at risk (OARs), which limits their prediction efficiency and application potential. To address this issue, we design a novel approach named as TransDose for dose distribution prediction that treats CT images as the unique input in this paper. Specifically, instead of inputting the segmentation masks to provide the prior anatomical information, we utilize a super-pixel-based graph convolutional network (GCN) to extract category-specific features, thereby compensating the network for the necessary anatomical knowledge. Besides, considering the strong continuous dependency between adjacent CT slices as well as adjacent dose maps, we embed the Transformer into the backbone, and make use of its superior ability of long-range sequence modeling to endow input features with inter-slice continuity message. To our knowledge, this is the first network that specially designed for the task of dose prediction from only CT images without ignoring necessary anatomical structure. Finally, we evaluate our model on two real datasets, and extensive experiments demonstrate the generalizability and advantages of our method.

Dose distribution prediction for head-and-neck cancer radiotherapy using a generative adversarial network: influence of input data

Purpose

A three-dimensional deep generative adversarial network (GAN) was used to predict dose distributions for locally advanced head and neck cancer radiotherapy. Given the labor- and time-intensive nature of manual planning target volume (PTV) and organ-at-risk (OAR) segmentation, we investigated whether dose distributions could be predicted without the need for fully segmented datasets.

Materials and methods

GANs were trained/validated/tested using 320/30/35 previously segmented CT datasets and treatment plans. The following input combinations were used to train and test the models: CT-scan only (C); CT+PTVboost/elective (CP); CT+PTVs+OARs+body structure (CPOB); PTVs+OARs+body structure (POB); PTVs+body structure (PB). Mean absolute errors (MAEs) for the predicted dose distribution and mean doses to individual OARs (individual salivary glands, individual swallowing structures) were analyzed.

Results

For the five models listed, MAEs were 7.3 Gy, 3.5 Gy, 3.4 Gy, 3.4 Gy, and 3.5 Gy, respectively, without significant differences among CP-CPOB, CP-POB, CP-PB, among CPOB-POB. Dose volume histograms showed that all four models that included PTV contours predicted dose distributions that had a high level of agreement with clinical treatment plans. The best model CPOB and the worst model PB (except model C) predicted mean dose to within ±3 Gy of the clinical dose, for 82.6%/88.6%/82.9% and 71.4%/67.1%/72.2% of all OARs, parotid glands (PG), and submandibular glands (SMG), respectively. The R2 values (0.17/0.96/0.97/0.95/0.95) of OAR mean doses for each model also indicated that except for model C, the predictions correlated highly with the clinical dose distributions. Interestingly model C could reasonably predict the dose in eight patients, but on average, it performed inadequately.

Conclusion

We demonstrated the influence of the CT scan, and PTV and OAR contours on dose prediction. Model CP was not statistically different from model CPOB and represents the minimum data statistically required to adequately predict the clinical dose distribution in a group of patients.

Generating synthetic mixed-type longitudinal electronic health records for artificial intelligent applications

The recent availability of electronic health records (EHRs) have provided enormous opportunities to develop artificial intelligence (AI) algorithms. However, patient privacy has become a major concern that limits data sharing across hospital settings and subsequently hinders the advances in AI. Synthetic data, which benefits from the development and proliferation of generative models, has served as a promising substitute for real patient EHR data. However, the current generative models are limited as they only generate single type of clinical data for a synthetic patient, i.e., either continuous-valued or discrete-valued. To mimic the nature of clinical decision-making which encompasses various data types/sources, in this study, we propose a generative adversarial network (GAN) entitled EHR-M-GAN that simultaneously synthesizes mixed-type timeseries EHR data. EHR-M-GAN is capable of capturing the multidimensional, heterogeneous, and correlated temporal dynamics in patient trajectories. We have validated EHR-M-GAN on three publicly-available intensive care unit databases with records from a total of 141,488 unique patients, and performed privacy risk evaluation of the proposed model. EHR-M-GAN has demonstrated its superiority over state-of-the-art benchmarks for synthesizing clinical timeseries with high fidelity, while addressing the limitations regarding data types and dimensionality in the current generative models. Notably, prediction models for outcomes of intensive care performed significantly better when training data was augmented with the addition of EHR-M-GAN-generated timeseries. EHR-M-GAN may have use in developing AI algorithms in resource-limited settings, lowering the barrier for data acquisition while preserving patient privacy.

Data synthesis and adversarial networks: A review and meta-analysis in cancer imaging

Despite technological and medical advances, the detection, interpretation, and treatment of cancer based on imaging data continue to pose significant challenges. These include inter-observer variability, class imbalance, dataset shifts, inter- and intra-tumour heterogeneity, malignancy determination, and treatment effect uncertainty. Given the recent advancements in image synthesis, Generative Adversarial Networks (GANs), and adversarial training, we assess the potential of these technologies to address a number of key challenges of cancer imaging. We categorise these challenges into (a) data scarcity and imbalance, (b) data access and privacy, (c) data annotation and segmentation, (d) cancer detection and diagnosis, and (e) tumour profiling, treatment planning and monitoring. Based on our analysis of 164 publications that apply adversarial training techniques in the context of cancer imaging, we highlight multiple underexplored solutions with research potential. We further contribute the Synthesis Study Trustworthiness Test (SynTRUST), a meta-analysis framework for assessing the validation rigour of medical image synthesis studies. SynTRUST is based on 26 concrete measures of thoroughness, reproducibility, usefulness, scalability, and tenability. Based on SynTRUST, we analyse 16 of the most promising cancer imaging challenge solutions and observe a high validation rigour in general, but also several desirable improvements. With this work, we strive to bridge the gap between the needs of the clinical cancer imaging community and the current and prospective research on data synthesis and adversarial networks in the artificial intelligence community.

3D dose prediction for Gamma Knife radiosurgery using deep learning and data modification

PURPOSE

To develop a machine learning-based, 3D dose prediction methodology for Gamma Knife (GK) radiosurgery. The methodology accounts for cases involving targets of any number, size, and shape.

METHODS

Data from 322 GK treatment plans was modified by isolating and cropping the contoured MRI and clinical dose distributions based on tumor location, then scaling the resulting tumor spaces to a standard size. An accompanying 3D tensor was created for each instance to account for tumor size. The modified dataset for 272 patients was used to train both a generative adversarial network (GAN-GK) and a 3D U-Net model (U-Net-GK). Unmodified data was used to train equivalent baseline models. All models were used to predict the dose distribution of 50 out-of-sample patients. Prediction accuracy was evaluated using gamma, with criteria of 4 %/2mm, 3 %/3mm, 3 %/1mm and 1 %/1mm. Prediction quality was assessed using coverage, selectivity, and conformity indices.

RESULTS

The predictions resulting from GAN-GK and U-Net-GK were similar to their clinical counterparts, with average gamma (4 %/2mm) passing rates of 84.9 ± 15.3 % and 83.1 ± 17.2 %, respectively. In contrast, the gamma passing rate of baseline models were significantly worse than their respective GK-specific models (p < 0.001) at all criterion levels. The quality of GK-specific predictions was also similar to that of clinical plans.

CONCLUSION

Deep learning models can use GK-specific data modification to predict 3D dose distributions for GKRS plans with a large range in size, shape, or number of targets. Standard deep learning models applied to unmodified GK data generated poorer predictions.

Can the use of knowledge-based planning systems improve stereotactic radiotherapy planning? A systematic review

Introduction:

This study aimed to systematically review the literature to synthesise and summarise whether using knowledge-based planning (KBP) can improve the planning of stereotactic radiotherapy treatments.

Methods:

A systematic literature search was carried out using Medline, Scopus and Cochrane databases to evaluate the use of KBP planning in stereotactic radiotherapy. Three hundred twenty-five potential studies were identified and screened to find 25 relevant studies.

Results:

Twenty-five studies met the inclusion criteria. Where a commercial KBP was used, 72.7% of studies reported a quality improvement, and 45.5% reported a reduction in planning time. There is evidence that when used as a quality control tool, KBP can highlight stereotactic plans that need revision. In studies that use KBP as the starting point for radiotherapy planning optimisation, the radiotherapy plans generated are typically equal to or superior to those planned manually.

Conclusions:

There is evidence that KBP has the potential to improve the quality and speed of stereotactic radiotherapy planning. Further research is required to accurately quantify such systems’ quality improvements and time savings. Notably, there has been little research into their use for prostate, spinal or liver stereotactic radiotherapy, and research in these areas would be desirable. It is recommended that future studies use the ICRU 91 level 2 reporting format and that blinded physician review could add a qualitative assessment of KBP system performance.

Small beams, fast predictions: a comparison of machine learning dose prediction models for proton minibeam therapy

BACKGROUND

Dose calculations for novel radiotherapy cancer treatments such as proton minibeam radiation therapy is often done using full Monte Carlo (MC) simulations. As MC simulations can be very time consuming for this kind of application, deep learning models have been considered to accelerate dose estimation in cancer patients.

PURPOSE

This work systematically evaluates the dose prediction accuracy, speed and generalization performance of three selected state-of-the-art deep learning models for dose prediction applied to the proton minibeam therapy. The strengths and weaknesses of those models are thoroughly investigated, helping other researchers to decide on a viable algorithm for their own application.

METHODS

The following recently published models are compared: first, a 3D U-Net model trained as a regression network, second, a 3D U-Net trained as a generator of a generative adversarial network (GAN) and third, a dose transformer model which interprets the dose prediction as a sequence translation task. These models are trained to emulate the result of MC simulations. The dose depositions of a proton minibeam with a diameter of 800μm and an energy of 20-100 MeV inside a simple head phantom calculated by full Geant4 MC simulations are used as a case study for this comparison. The spatial resolution is 0.5 mm. Special attention is put on the evaluation of the generalization performance of the investigated models.

RESULTS

Dose predictions with all models are produced in the order of a second on a GPU, the 3D U-Net models being fastest with an average of 130 ms. An investigated 3D U-Net regression model is found to show the strongest performance with overall 61.0 % ± $\%\pm$ 0.5% of all voxels exhibiting a deviation in energy deposition prediction of less than 3% compared to full MC simulations with no spatial deviation allowed. The 3D U-Net models are observed to show better generalization performance for target geometry variations, while the transformer-based model shows better generalization with regard to the proton energy.

CONCLUSIONS

This paper reveals that (1) all studied deep learning models are significantly faster than non-machine learning approaches predicting the dose in the order of seconds compared to hours for MC, (2) all models provide reasonable accuracy, and (3) the regression-trained 3D U-Net provides the most accurate predictions.

Applications of artificial intelligence in stereotactic body radiation therapy

This topical review focuses on the applications of artificial intelligence (AI) tools to stereotactic body radiation therapy (SBRT). The high dose per fraction and the limited number of fractions in SBRT require stricter accuracy than standard radiation therapy. The intent of this review is to describe the development and evaluate the possible benefit of AI tools integration into the radiation oncology workflow for SBRT automation. The selected papers were subdivided into four sections, representative of the whole radiotherapy process: ‘AI in SBRT target and organs at risk contouring’, ‘AI in SBRT planning’, ‘AI during the SBRT delivery’, and ‘AI for outcome prediction after SBRT’. Each section summarises the challenges, as well as limits and needs for improvement to achieve better integration of AI tools in the clinical workflow.

Artificial intelligence in radiotherapy

Radiotherapy is a discipline closely integrated with computer science. Artificial intelligence (AI) has developed rapidly over the past few years. With the explosive growth of medical big data, AI promises to revolutionize the field of radiotherapy through highly automated workflow, enhanced quality assurance, improved regional balances of expert experiences, and individualized treatment guided by multi-omics. In addition to independent researchers, the increasing number of large databases, biobanks, and open challenges significantly facilitated AI studies on radiation oncology. This article reviews the latest research, clinical applications, and challenges of AI in each part of radiotherapy including image processing, contouring, planning, quality assurance, motion management, and outcome prediction. By summarizing cutting-edge findings and challenges, we aim to inspire researchers to explore more future possibilities and accelerate the arrival of AI radiotherapy.

CT-Only Radiotherapy: An Exploratory Study for Automatic Dose Prediction on Rectal Cancer Patients Via Deep Adversarial Network

Purpose

Current deep learning methods for dose prediction require manual delineations of planning target volume (PTV) and organs at risk (OARs) besides the original CT images. Perceiving the time cost of manual contour delineation, we expect to explore the feasibility of accelerating the radiotherapy planning by leveraging only the CT images to produce high-quality dose distribution maps while generating the contour information automatically.

Materials and Methods

We developed a generative adversarial network (GAN) with multi-task learning (MTL) strategy to produce accurate dose distribution maps without manually delineated contours. To balance the relative importance of each task (i.e., the primary dose prediction task and the auxiliary tumor segmentation task), a multi-task loss function was employed. Our model was trained, validated and evaluated on a cohort of 130 rectal cancer patients.

Results

Experimental results manifest the feasibility and improvements of our contour-free method. Compared to other mainstream methods (i.e., U-net, DeepLabV3+, DoseNet, and GAN), the proposed method produces the leading performance with statistically significant improvements by achieving the highest HI of 1.023 (3.27E-5) and the lowest prediction error with ΔD95 of 0.125 (0.035) and ΔDmean of 0.023 (4.19E-4), respectively. The DVH differences between the predicted dose and the ideal dose are subtle and the errors in the difference maps are minimal. In addition, we conducted the ablation study to validate the effectiveness of each module. Furthermore, the results of attention maps also prove that our CT-only prediction model is capable of paying attention to both the target tumor (i.e., high dose distribution area) and the surrounding healthy tissues (i.e., low dose distribution areas).

Conclusion

The proposed CT-only dose prediction framework is capable of producing acceptable dose maps and reducing the time and labor for manual delineation, thus having great clinical potential in providing accurate and accelerated radiotherapy. Code is available at https://github.com/joegit-code/DoseWithCT

An automatic method to generate voxel-based absorbed doses from radioactivity distributions for nuclear medicine using generative adversarial networks: a feasibility study

An approach to autogenerate voxel-based absorbed dose for nuclear medicine is proposed using generative adversarial networks. The method is based on image-to-image transformation and promises to achieve real-time visualization of the absorbed dose and optimization of therapeutic strategies. The activity-density superimposed image is input to generator (G) as a reference image to generate a pseudoabsorbed dose image (DI), which is then mixed with ground truth (GT) DI and recognized by discriminator (D). If the pseudoimage is recognized, the information is fed back, and G regenerates a pseudodose image until D drops to obtain a lifelike DI. As a feasibility study, we used the dose distribution of segmented human anatomy from different sources and activities as training and test datasets. The activity source was assumed to be 1, 2, 3, 4, or 7 subsource blocks. The 3-subsource model was used as the test dataset, and the others were used as the training dataset. The activity distribution in the subsource was assumed to be uniform or heterogeneous (i.e., Gaussian diffusion with sigma 0.0, 0.3, or 0.6). Differences were assessed by Gamma analysis. Results showed that the same or quasi-inhomogeneity model can well predict the dose distribution of different activity-inhomogeneity. Although the 1-source model was trained with very few datasets, it showed an optimal balance between accuracy and training efficiency. There were offsets in the mean absorbed dose between the predicted and GT, but they all showed a higher Gamma-pass-rate (> 93%) and ~ 10% std.

A generative adversarial inpainting network to enhance prediction of periodontal clinical attachment level

OBJECTIVES

Bone level as measured by clinical attachment levels (CAL) are critical findings that determine the diagnosis of periodontal disease. Deep learning algorithms are being used to determine CAL which aid in the diagnosis of periodontal disease. However, the limited field-of-view of bitewing x-rays poses a challenge for convolutional neural networks (CNN) because out-of-view anatomy cannot be directly considered. This study presents an inpainting algorithm using generative adversarial networks (GANs) coupled with partial convolutions to predict out-of-view anatomy to enhance CAL prediction accuracy.

METHODS

Retrospective purposive sampling of cases with healthy periodontium and diseased periodontium with bitewing and periapical radiographs and clinician recorded CAL were utilized. Data utilized was from July 1, 2016 through January 30, 2020. 80,326 images were used for training, 12,901 images were used for validation and 10,687 images were used to compare non-inpainted methods to inpainted methods for CAL predictions. Statistical analyses were mean bias error (MBE), mean absolute error (MAE) and Dunn's pairwise test comparing CAL at p=0.05.

RESULTS

Comparator p-values demonstrated statistically significant improvement in CAL prediction accuracy between corresponding inpainted and non-inpainted methods with MAE of 1.04 mm and 1.50 mm respectively. The Dunn's pairwise test indicated statistically significant improvement in CAL prediction accuracy between inpainted methods compared to their non-inpainted counterparts, with the best performing methods achieving a Dunn's pairwise value of -63.89.

CONCLUSIONS

This study demonstrates the superiority of using a generative adversarial inpainting network with partial convolutions to predict CAL from bitewing and periapical images.

CLINICAL SIGNIFICANCE

Artificial intelligence was developed and utilized to predict clinical attachment level compared to clinical measurements. A generative adversarial inpainting network with partial convolutions was developed, tested and validated to predict clinical attachment level. The inpainting approach was found to be superior to non-inpainted methods and within the 1mm clinician-determined measurement standard.

A Survey on Deep Learning for Precision Oncology

Precision oncology, which ensures optimized cancer treatment tailored to the unique biology of a patient’s disease, has rapidly developed and is of great clinical importance. Deep learning has become the main method for precision oncology. This paper summarizes the recent deep-learning approaches relevant to precision oncology and reviews over 150 articles within the last six years. First, we survey the deep-learning approaches categorized by various precision oncology tasks, including the estimation of dose distribution for treatment planning, survival analysis and risk estimation after treatment, prediction of treatment response, and patient selection for treatment planning. Secondly, we provide an overview of the studies per anatomical area, including the brain, bladder, breast, bone, cervix, esophagus, gastric, head and neck, kidneys, liver, lung, pancreas, pelvis, prostate, and rectum. Finally, we highlight the challenges and discuss potential solutions for future research directions.

Towards a safe and efficient clinical implementation of machine learning in radiation oncology by exploring model interpretability, explainability and data-model dependency

The interest in machine learning (ML) has grown tremendously in recent years, partly due to the performance leap that occurred with new techniques of deep learning, convolutional neural networks for images, increased computational power, and wider availability of large datasets. Most fields of medicine follow that popular trend and, notably, radiation oncology is one of those that are at the forefront, with already a long tradition in using digital images and fully computerized workflows. ML models are driven by data, and in contrast with many statistical or physical models, they can be very large and complex, with countless generic parameters. This inevitably raises two questions, namely, the tight dependence between the models and the datasets that feed them, and the interpretability of the models, which scales with its complexity. Any problems in the data used to train the model will be later reflected in their performance. This, together with the low interpretability of ML models, makes their implementation into the clinical workflow particularly difficult. Building tools for risk assessment and quality assurance of ML models must involve then two main points: interpretability and data-model dependency. After a joint introduction of both radiation oncology and ML, this paper reviews the main risks and current solutions when applying the latter to workflows in the former. Risks associated with data and models, as well as their interaction, are detailed. Next, the core concepts of interpretability, explainability, and data-model dependency are formally defined and illustrated with examples. Afterwards, a broad discussion goes through key applications of ML in workflows of radiation oncology as well as vendors' perspectives for the clinical implementation of ML.

Attention-aware 3D U-Net convolutional neural network for knowledge-based planning 3D dose distribution prediction of head-and-neck cancer

PURPOSE

Deep learning-based knowledge-based planning (KBP) methods have been introduced for radiotherapy dose distribution prediction to reduce the planning time and maintain consistent high-quality plans. This paper presents a novel KBP model using an attention-gating mechanism and a three-dimensional (3D) U-Net for intensity-modulated radiation therapy (IMRT) 3D dose distribution prediction in head-and-neck cancer.

METHODS

A total of 340 head-and-neck cancer plans, representing the OpenKBP-2020 AAPM Grand Challenge data set, were used in this study. All patients were treated with the IMRT technique and a dose prescription of 70 Gy. The data set was randomly divided into 64%/16%/20% as training/validation/testing cohorts. An attention-gated 3D U-Net architecture model was developed to predict full 3D dose distribution. The developed model was trained using the mean-squared error loss function, Adam optimization algorithm, a learning rate of 0.001, 120 epochs, and batch size of 4. In addition, a baseline U-Net model was also similarly trained for comparison. The model performance was evaluated on the testing data set by comparing the generated dose distributions against the ground-truth dose distributions using dose statistics and clinical dosimetric indices. Its performance was also compared to the baseline model and the reported results of other deep learning-based dose prediction models.

RESULTS

The proposed attention-gated 3D U-Net model showed high capability in accurately predicting 3D dose distributions that closely replicated the ground-truth dose distributions of 68 plans in the test set. The average value of the mean absolute dose error was 2.972 ± 1.220 Gy (vs. 2.920 ± 1.476 Gy for a baseline U-Net) in the brainstem, 4.243 ± 1.791 Gy (vs. 4.530 ± 2.295 Gy for a baseline U-Net) in the left parotid, 4.622 ± 1.975 Gy (vs. 4.223 ± 1.816 Gy for a baseline U-Net) in the right parotid, 3.346 ± 1.198 Gy (vs. 2.958 ± 0.888 Gy for a baseline U-Net) in the spinal cord, 6.582 ± 3.748 Gy (vs. 5.114 ± 2.098 Gy for a baseline U-Net) in the esophagus, 4.756 ± 1.560 Gy (vs. 4.992 ± 2.030 Gy for a baseline U-Net) in the mandible, 4.501 ± 1.784 Gy (vs. 4.925 ± 2.347 Gy for a baseline U-Net) in the larynx, 2.494 ± 0.953 Gy (vs. 2.648 ± 1.247 Gy for a baseline U-Net) in the PTV_70, and 2.432 ± 2.272 Gy (vs. 2.811 ± 2.896 Gy for a baseline U-Net) in the body contour. The average difference in predicting the D₉₉ value for the targets (PTV_70, PTV_63, and PTV_56) was 2.50 ± 1.77 Gy. For the organs at risk, the average difference in predicting the D m a x ${D_{max}}$ (brainstem, spinal cord, and mandible) and D m e a n ${D_{mean}}$ (left parotid, right parotid, esophagus, and larynx) values was 1.43 ± 1.01 and 2.44 ± 1.73 Gy, respectively. The average value of the homogeneity index was 7.99 ± 1.45 for the predicted plans versus 5.74 ± 2.95 for the ground-truth plans, whereas the average value of the conformity index was 0.63 ± 0.17 for the predicted plans versus 0.89 ± 0.19 for the ground-truth plans. The proposed model needs less than 5 s to predict a full 3D dose distribution of 64 × 64 × 64 voxels for a new patient that is sufficient for real-time applications.

CONCLUSIONS

The attention-gated 3D U-Net model demonstrated a capability in predicting accurate 3D dose distributions for head-and-neck IMRT plans with consistent quality. The prediction performance of the proposed model was overall superior to a baseline standard U-Net model, and it was also competitive to the performance of the best state-of-the-art dose prediction method reported in the literature. The proposed model could be used to obtain dose distributions for decision-making before planning, quality assurance of planning, and guiding-automated planning for improved plan consistency, quality, and planning efficiency.

Machine Learning Applications of Surgical Imaging for the Diagnosis and Treatment of Spine Disorders: Current State of the Art

Recent developments in machine learning (ML) methods demonstrate unparalleled potential for application in the spine. The ability for ML to provide diagnostic faculty, produce novel insights from existing capabilities, and augment or accelerate elements of surgical planning and decision making at levels equivalent or superior to humans will tremendously benefit spine surgeons and patients alike. In this review, we aim to provide a clinically relevant outline of ML-based technology in the contexts of spinal deformity, degeneration, and trauma, as well as an overview of commercial-level and precommercial-level surgical assist systems and decisional support tools. Furthermore, we briefly discuss potential applications of generative networks before highlighting some of the limitations of ML applications. We conclude that ML in spine imaging represents a significant addition to the neurosurgeon's armamentarium-it has the capacity to directly address and manifest clinical needs and improve diagnostic and procedural quality and safety-but is yet subject to challenges that must be addressed before widespread implementation.

Multi-constraint generative adversarial network for dose prediction in radiotherapy

Radiation therapy (RT) is regarded as the primary treatment for cancer in the clinic, aiming to deliver an accurate dose to the planning target volume (PTV) while protecting the surrounding organs at risk (OARs). To improve the effectiveness of the treatment planning, deep learning methods are widely adopted to predict dose distribution maps for clinical treatment planning. In this paper, we present a novel multi-constraint dose prediction model based on generative adversarial network, named Mc-GAN, to automatically predict the dose distribution map from the computer tomography (CT) images and the masks of PTV and OARs. Specifically, the generator is an embedded UNet-like structure with dilated convolution to capture both the global and local information. During the feature extraction, a dual attention module (DAM) is embedded to force the generator to take more heed of internal semantic relevance. To improve the prediction accuracy, two additional losses, i.e., the locality-constrained loss (LCL) and the self-supervised perceptual loss (SPL), are introduced besides the conventional global pixel-level loss and adversarial loss. Concretely, the LCL tries to focus on the predictions of locally important areas while the SPL aims to prevent the predicted dose maps from the possible distortion at the feature level. Evaluated on two in-house datasets, our proposed Mc-GAN has been demonstrated to outperform other state-of-the-art methods in almost all PTV and OARs criteria.

Dose Prediction Using a Three-Dimensional Convolutional Neural Network for Nasopharyngeal Carcinoma With Tomotherapy

Purpose

This study focused on predicting 3D dose distribution at high precision and generated the prediction methods for nasopharyngeal carcinoma patients (NPC) treated with Tomotherapy based on the patient-specific gap between organs at risk (OARs) and planning target volumes (PTVs).

Methods

A convolutional neural network (CNN) is trained using the CT and contour masks as the input and dose distributions as output. The CNN is based on the "3D Dense-U-Net", which combines the U-Net and the Dense-Net. To evaluate the model, we retrospectively used 124 NPC patients treated with Tomotherapy, in which 96 and 28 patients were randomly split and used for model training and test, respectively. We performed comparison studies using different training matrix shapes and dimensions for the CNN models, i.e., 128 ×128 ×48 (for Model I), 128 ×128 ×16 (for Model II), and 2D Dense U-Net (for Model III). The performance of these models was quantitatively evaluated using clinically relevant metrics and statistical analysis.

Results

We found a more considerable height of the training patch size yields a better model outcome. The study calculated the corresponding errors by comparing the predicted dose with the ground truth. The mean deviations from the mean and maximum doses of PTVs and OARs were 2.42 and 2.93%. Error for the maximum dose of right optic nerves in Model I was 4.87 ± 6.88%, compared with 7.9 ± 6.8% in Model II (p=0.08) and 13.85 ± 10.97% in Model III (p<0.01); the Model I performed the best. The gamma passing rates of PTV₆₀ for 3%/3 mm criteria was 83.6 ± 5.2% in Model I, compared with 75.9 ± 5.5% in Model II (p<0.001) and 77.2 ± 7.3% in Model III (p<0.01); the Model I also gave the best outcome. The prediction error of D₉₅ for PTV₆₀ was 0.64 ± 0.68% in Model I, compared with 2.04 ± 1.38% in Model II (p<0.01) and 1.05 ± 0.96% in Model III (p=0.01); the Model I was also the best one.

Conclusions

It is significant to train the dose prediction model by exploiting deep-learning techniques with various clinical logic concepts. Increasing the height (Y direction) of training patch size can improve the dose prediction accuracy of tiny OARs and the whole body. Our dose prediction network model provides a clinically acceptable result and a training strategy for a dose prediction model. It should be helpful to build automatic Tomotherapy planning.

Artificial intelligence applications in intensity modulated radiation treatment planning: an overview

Artificial intelligence (AI) refers to methods that improve and automate challenging human tasks by systematically capturing and applying relevant knowledge in these tasks. Over the past decades, a number of approaches have been developed to address different types and needs of system intelligence ranging from search strategies to knowledge representation and inference to robotic planning. In the context of radiation treatment planning, multiple AI approaches may be adopted to improve the planning quality and efficiency. For example, knowledge representation and inference methods may improve dose prescription by integrating and reasoning about the domain knowledge described in many clinical guidelines and clinical trials reports. In this review, we will focus on the most studied AI approach in intensity modulated radiation therapy (IMRT)/volumetric modulated arc therapy (VMAT)-machine learning (ML) and describe our recent efforts in applying ML to improve the quality, consistency, and efficiency of IMRT/VMAT planning. With the available high-quality data, we can build models to accurately predict critical variables for each step of the planning process and thus automate and improve its outcomes. Specific to the IMRT/VMAT planning process, we can build models for each of the four critical components in the process: dose-volume histogram (DVH), Dose, Fluence, and Human Planner. These models can be divided into two general groups. The first group focuses on encoding prior experience and knowledge through ML and more recently deep learning (DL) from prior clinical plans and using these models to predict the optimal DVH (DVH prediction model), or 3D dose distribution (dose prediction model), or fluence map (fluence map model). The goal of these models is to reduce or remove the trial-and-error process and guarantee consistently high-quality plans. The second group of models focuses on mimicking human planners' decision-making process (planning strategy model) during the iterative adjustments/guidance of the optimization engine. Each critical step of the IMRT/VMAT treatment planning process can be improved and automated by AI methods. As more training data becomes available and more sophisticated models are developed, we can expect that the AI methods in treatment planning will continue to improve accuracy, efficiency, and robustness.

Knowledge-based radiation treatment planning: A data-driven method survey

This paper surveys the data-driven dose prediction methods investigated for knowledge-based planning (KBP) in the last decade. These methods were classified into two major categories-traditional KBP methods and deep-learning (DL) methods-according to their techniques of utilizing previous knowledge. Traditional KBP methods include studies that require geometric or anatomical features to either find the best-matched case(s) from a repository of prior treatment plans or to build dose prediction models. DL methods include studies that train neural networks to make dose predictions. A comprehensive review of each category is presented, highlighting key features, methods, and their advancements over the years. We separated the cited works according to the framework and cancer site in each category. Finally, we briefly discuss the performance of both traditional KBP methods and DL methods, then discuss future trends of both data-driven KBP methods to dose prediction.

Technical Note: Dose prediction for radiation therapy using feature-based losses and One Cycle Learning

Purpose

To present the technical details of the runner‐up model in the open knowledge‐based planning (OpenKBP) challenge for the dose–volume histogram (DVH) stream. The model was designed to ensure simple and reproducible training, without the necessity of costly advanced generative adversarial network (GAN) techniques.

Methods

The model was developed based on the OpenKBP challenge dataset, consisting of 200 and 40 head‐and‐neck patients for training and validation, respectively. The final model is a U‐Net with additional ResNet blocks between up‐ and down convolutions. The results were obtained by training the model with AdamW with the One Cycle scheduler. The loss function is a combination of the L1 loss with a feature loss, which uses a pretrained video classifier as a feature extractor. The performance was evaluated on another 100 patients in the OpenKBP test dataset. The DVH metrics of the test data were evaluated, where D0.1cc, and Dmean were calculated for the organs at risk (OARs) and D1%, D95%, and D99% were computed for the target structures. DVH metric differences between predicted and true dose are reported in percentage.

Results

The model achieved 2nd and 4th place in the DVH and dose stream of the OpenKBP challenge, respectively. The dose and DVH score were 2.62 ± 1.10 and 1.52 ± 1.06, respectively. Mean dose differences for the different structures and DVH parameters were within ±1%.

Conclusion

This straightforward approach produced excellent results. It incorporated One Cycle Learning, ResNet, and feature‐based losses, which are common computer vision techniques.

Learning-based dose prediction for pancreatic stereotactic body radiation therapy using dual pyramid adversarial network

Treatment planning for pancreatic cancer stereotactic body radiation therapy (SBRT) is very challenging owing to vast spatial variations and close proximity of many organs-at-risk. Recently, deep learning (DL) based methods have been applied in dose prediction tasks of various treatment sites with the aim of relieving planning challenges. However, its effectiveness on pancreatic cancer SBRT is yet to be fully explored due to limited investigations in the literature. This study aims to further current knowledge in DL-based dose prediction tasks by implementing and demonstrating the feasibility of a new dual pyramid networks (DPNs) integrated DL model for predicting dose distributions of pancreatic SBRT. The proposed framework is composed of four parts: CT-only feature pyramid network (FPN), contour-only FPN, late fusion network and an adversarial network. During each phase of the network, combination of mean absolute error, gradient difference error, histogram matching, and adversarial loss is used for supervision. The performance of proposed model was demonstrated for pancreatic cancer SBRT plans with doses prescribed between 33 and 50 Gy across as many as three planning target volumes (PTVs) in five fractions. Five-fold cross validation was performed on 30 patients, and another 20 patients were used as holdout tests of trained model. Predicted plans were compared with clinically approved plans through dose volume parameters and two-paired t-test. For the same sets, our results were compared with three different DL architectures: 3D U-Net, 3D U-Net with adversarial learning, and DPN without adversarial learning. The proposed framework was able to predict 87% and 91% of clinically relevant dose parameters for cross validation sets and holdout sets, respectively, without any significant differences (P > 0.05). Dose distribution predicted by our framework was also able to predict the intentional hotspots as feature characteristics of SBRT plans. Our method achieved higher correlation coefficients with the ground truth in 22/26, 24/26 and 20/26 dose volume parameters compared to the network without adversarial learning, 3D U-Net, and 3D U-Net with adversarial learning, respectively. Overall, the proposed model was able to predict doses to cases with both single and multiple PTVs. In conclusion, the DPN integrated DL model was successfully implemented, and demonstrated good dose prediction accuracy and dosimetric characteristics for pancreatic cancer SBRT.

Generating Synthetic Fermentation Data of Shindari, a Traditional Jeju Beverage, Using Multiple Imputation Ensemble and Generative Adversarial Networks

Fermentation is an age-old technique used to preserve food by restoring proper microbial balance. Boiled barley and nuruk are fermented for a short period to produce Shindari, a traditional beverage for the people of Jeju, South Korea. Shindari has been proven to be a drink of multiple health benefits if fermented for an optimal period. It is necessary to predict the ideal fermentation time required by each microbial community to keep the advantages of the microorganisms produced by the fermentation process in Shindari intact and to eliminate contamination. Prediction through machine learning requires past data but the process of obtaining fermentation data of Shindari is time consuming, expensive, and not easily available. Therefore, there is a need to generate synthetic fermentation data to explore various benefits of the drink and to reduce any risk from overfermentation. In this paper, we propose a model that takes incomplete tabular fermentation data of Shindari as input and uses multiple imputation ensemble (MIE) and generative adversarial networks (GAN) to generate synthetic fermentation data that can be later used for prediction and microbial spoilage control. For multiple imputation, we used multivariate imputation by chained equations and random forest imputation, and ensembling was done using the bagging and stacking method. For generating synthetic data, we remodeled the tabular GAN with skip connections and adapted the architecture of Wasserstein GAN with gradient penalty. We compared the performance of our model with other imputation and ensemble models using various evaluation metrics and visual representations. Our GAN model could overcome the mode collapse problem and converged at a faster rate than existing GAN models for synthetic data generation. Experiment results show that our proposed model executes with less error, is more accurate, and generates significantly better synthetic fermentation data compared to other models.

Dose calculation in proton therapy using a discovery cross-domain generative adversarial network (DiscoGAN)

PURPOSE

Accurate dose calculation is a critical step in proton therapy. A novel machine learning-based approach was proposed to achieve comparable accuracy to that of Monte Carlo simulation while reducing the computational time.

METHODS

Computed tomography-based patient phantoms were used and three treatment sites were selected (thorax, head, and abdomen), comprising different beam pathways and beam energies. The training data were generated using Monte Carlo simulations. A discovery cross-domain generative adversarial network (DiscoGAN) was developed to perform the mapping between two domains: stopping power and dose, with HU values from CT images incorporated as auxiliary features. The accuracy of dose calculation was quantitatively evaluated in terms of mean relative error (MRE) and mean absolute error (MAE). The relationship between the DiscoGAN performance and other factors such as absolute dose, beam energy and location within the beam cross-section (center and off-center lines) was examined.

RESULTS

The DiscoGAN model is found to be effective in dose calculation. For the abdominal case, the MRE is found to 1.47% (mean), 3.30% (maximum) and 0.67% (minimum). For the thoracic case, the MRE is found to ~2.43% (mean), 4.80% (maximum) and 0.71% (minimum). For the head case, the MRE is found to ~2.83% (mean), 4.84% (maximum) and 1.01% (minimum). Comparable accuracy is found in the independent validation dataset (different CT images), achieving a mean MRE of ~1.65% (thorax), 4.02% (head) and 1.64% (abdomen). For the energy span between 80 and 130 MeV, no strong dependency of accuracy on beam energy is found. The results imply that no systematic deviation, either over-dose or under-dose, occurs between the predicted dose and raw dose.

CONCLUSION

The DiscoGAN framework demonstrates great potential as a tool for dose calculation in proton therapy, achieving comparable accuracy yet being more efficient relative to Monte Carlo simulation. Its comparison with the pencil beam algorithm (PBA) will be the next step of our research. If successful, our proposed approach is expected to find its use in more advanced applications such as inverse planning and adaptive proton therapy.

Artificial intelligence and machine learning for medical imaging: A technology review

Artificial intelligence (AI) has recently become a very popular buzzword, as a consequence of disruptive technical advances and impressive experimental results, notably in the field of image analysis and processing. In medicine, specialties where images are central, like radiology, pathology or oncology, have seized the opportunity and considerable efforts in research and development have been deployed to transfer the potential of AI to clinical applications. With AI becoming a more mainstream tool for typical medical imaging analysis tasks, such as diagnosis, segmentation, or classification, the key for a safe and efficient use of clinical AI applications relies, in part, on informed practitioners. The aim of this review is to present the basic technological pillars of AI, together with the state-of-the-art machine learning methods and their application to medical imaging. In addition, we discuss the new trends and future research directions. This will help the reader to understand how AI methods are now becoming an ubiquitous tool in any medical image analysis workflow and pave the way for the clinical implementation of AI-based solutions.

Automatic segmentation of three clinical target volumes in radiotherapy using lifelong learning

BACKGROUND AND PURPOSE

Convolutional neural networks (CNNs) have comparable human level performance in automatic segmentation. An important challenge that CNNs face in segmentation is catastrophic forgetting. They lose performance on tasks that were previously learned when trained on task. In this study, we propose a lifelong learning method to learn multiple segmentation tasks continuously without forgetting previous tasks.

MATERIALS AND METHODS

The cohort included three tumors, 800 patients of which had nasopharyngeal cancer (NPC), 800 patients had breast cancer, and 800 patients had rectal cancer. The tasks included segmentation of the clinical target volume (CTV) of these three cancers. The proposed lifelong learning network adopted dilation adapter to learn three segmentation tasks one by one. Only the newly added dilation adapter (seven layers) was fine tuning for incoming new task, whereas all the other learned layers were frozen.

RESULTS

Compared with single-task, multi-task or transfer learning, the proposed lifelong learning can achieve better or comparable segmentation accuracy with a DSC of 0.86 for NPC, 0.89 for breast cancer, and 0.87 for rectal cancer. Lifelong learning can avoid forgetting in sequential learning and yield good performance with less training data. Furthermore, it is more efficient than single-task or transfer learning, which reduced the number of parameters, size of model, and training time by ~58.8%, ~55.6%, and ~25.0%, respectively.

CONCLUSION

The proposed method preserved the knowledge of previous tasks while learning a new one using a dilation adapter. It could yield comparable performance with much less training data, model parameters, and training time.