Deep Learning for Radiotherapy Outcome Prediction Using Dose Data - A Review

RadDeploy: A framework for integrating in-house developed software and artificial intelligence models seamlessly into radiotherapy workflows

The use of and research in automation and artificial intelligence (AI) in radiotherapy is moving with incredible pace. Many innovations do, however, not make it into the clinic. One technical reason for this may be the lack of a platform to deploy such software into clinical practice. We suggest RadDeploy as a framework for integrating containerized software in clinical workflows outside of treatment planning systems. RadDeploy supports multiple DICOM as input for model containers and can run model containers asynchronously across GPUs and computers. This technical note summarizes the inner workings of RadDeploy and demonstrates three use-cases with varying complexity.

Longitudinal Image Data for Outcome Modeling

In oncology, medical imaging is crucial for diagnosis, treatment planning and therapy execution. Treatment responses can be complex and varied and are known to involve factors of treatment, patient characteristics and tumor microenvironment. Longitudinal image analysis is able to track temporal changes, aiding in disease monitoring, treatment evaluation, and outcome prediction. This allows for the enhancement of personalized medicine. However, analyzing longitudinal 2D and 3D images presents unique challenges, including image registration, reliable segmentation, dealing with variable imaging intervals, and sparse data. This review presents an overview of techniques and methodologies in longitudinal image analysis, with a primary focus on outcome modeling in radiation oncology.

Artificial Intelligence Uncertainty Quantification in Radiotherapy Applications - A Scoping Review

Background/purpose

The use of artificial intelligence (AI) in radiotherapy (RT) is expanding rapidly. However, there exists a notable lack of clinician trust in AI models, underscoring the need for effective uncertainty quantification (UQ) methods. The purpose of this study was to scope existing literature related to UQ in RT, identify areas of improvement, and determine future directions.

Methods

We followed the PRISMA-ScR scoping review reporting guidelines. We utilized the population (human cancer patients), concept (utilization of AI UQ), context (radiotherapy applications) framework to structure our search and screening process. We conducted a systematic search spanning seven databases, supplemented by manual curation, up to January 2024. Our search yielded a total of 8980 articles for initial review. Manuscript screening and data extraction was performed in Covidence. Data extraction categories included general study characteristics, RT characteristics, AI characteristics, and UQ characteristics.

Results

We identified 56 articles published from 2015-2024. 10 domains of RT applications were represented; most studies evaluated auto-contouring (50%), followed by image-synthesis (13%), and multiple applications simultaneously (11%). 12 disease sites were represented, with head and neck cancer being the most common disease site independent of application space (32%). Imaging data was used in 91% of studies, while only 13% incorporated RT dose information. Most studies focused on failure detection as the main application of UQ (60%), with Monte Carlo dropout being the most commonly implemented UQ method (32%) followed by ensembling (16%). 55% of studies did not share code or datasets.

Conclusion

Our review revealed a lack of diversity in UQ for RT applications beyond auto-contouring. Moreover, there was a clear need to study additional UQ methods, such as conformal prediction. Our results may incentivize the development of guidelines for reporting and implementation of UQ in RT.

A deep learning-based method for the prediction of temporal lobe injury in patients with nasopharyngeal carcinoma

PURPOSE

To establish a deep learning-based model to predict radiotherapy-induced temporal lobe injury (TLI).

MATERIALS AND METHODS

Spatial features of dose distribution within the temporal lobe were extracted using both the three-dimensional convolution (C3D) network and the dosiomics method. The Minimal Redundancy-Maximal-Relevance (mRMR) method was employed to rank the extracted features and select the most relevant ones. Four machine learning (ML) classifiers, including logistic regression (LR), k-nearest neighbors (kNN), support vector machines (SVM) and random forest (RF), were used to establish prediction models. Nested sampling and hyperparameter tuning methods were applied to train and validate the prediction models. For comparison, a prediction model base on the conventional D0.5cc of the temporal lobe obtained from dose volume (DV) histogram was established. The area under the receiver operating characteristic (ROC) curve (AUC) was utilized to compare the predictive performance of the different models.

RESULTS

A total of 127 nasopharyngeal carcinoma (NPC) patients were included in the study. In the model based on C3D deep learning features, the highest AUC value of 0.843 was achieved with 5 features. For the dosiomics features model, the highest AUC value of 0.715 was attained with 1 feature. Both of these models demonstrated superior performance compared to the prediction model based on DV parameters, which yielded an AUC of 0.695.

CONCLUSION

The prediction model utilizing C3D deep learning features outperformed models based on dosiomics features or traditional parameters in predicting the onset of TLI. This approach holds promise for predicting radiation-induced toxicities and guide individualized radiotherapy.

Dose-Incorporated Deep Ensemble Learning for Improving Brain Metastasis Stereotactic Radiosurgery Outcome Prediction

PURPOSE

To develop a novel deep ensemble learning model for accurate prediction of brain metastasis (BM) local control outcomes after stereotactic radiosurgery (SRS).

METHODS AND MATERIALS

A total of 114 brain metastases (BMs) from 82 patients were evaluated, including 26 BMs that developed biopsy-confirmed local failure post-SRS. The SRS spatial dose distribution (Dmap) of each BM was registered to the planning contrast-enhanced T1 (T1-CE) magnetic resonance imaging (MRI). Axial slices of the Dmap, T1-CE, and planning target volume (PTV) segmentation (PTVseg) intersecting the BM center were extracted within a fixed field of view determined by the 60% isodose volume in Dmap. A spherical projection was implemented to transform planar image content onto a spherical surface using multiple projection centers, and the resultant T1-CE/Dmap/PTVseg projections were stacked as a 3-channel variable. Four Visual Geometry Group (VGG-19) deep encoders were used in an ensemble design, with each submodel using a different spherical projection formula as input for BM outcome prediction. In each submodel, clinical features after positional encoding were fused with VGG-19 deep features to generate logit results. The ensemble's outcome was synthesized from the 4 submodel results via logistic regression. In total, 10 model versions with random validation sample assignments were trained to study model robustness. Performance was compared with (1) a single VGG-19 encoder, (2) an ensemble with a T1-CE MRI as the sole image input after projections, and (3) an ensemble with the same image input design without clinical feature inclusion.

RESULTS

The ensemble model achieved an excellent area under the receiver operating characteristic curve (AUCROC: 0.89 ± 0.02) with high sensitivity (0.82 ± 0.05), specificity (0.84 ± 0.11), and accuracy (0.84 ± 0.08) results. This outperformed the MRI-only VGG-19 encoder (sensitivity: 0.35 ± 0.01, AUCROC: 0.64 ± 0.08), the MRI-only deep ensemble (sensitivity: 0.60 ± 0.09, AUCROC: 0.68 ± 0.06), and the 3-channel ensemble without clinical feature fusion (sensitivity: 0.78 ± 0.08, AUCROC: 0.84 ± 0.03).

CONCLUSIONS

Facilitated by the spherical image projection method, a deep ensemble model incorporating Dmap and clinical variables demonstrated excellent performance in predicting BM post-SRS local failure. Our novel approach could improve other radiation therapy outcome models and warrants further evaluation.

A Review of Artificial Intelligence Application for Radiotherapy

Background and Purpose

Artificial intelligence (AI) is a technique which tries to think like humans and mimic human behaviors. It has been considered as an alternative in a lot of human-dependent steps in radiotherapy (RT), since the human participation is a principal uncertainty source in RT. The aim of this work is to provide a systematic summary of the current literature on AI application for RT, and to clarify its role for RT practice in terms of clinical views.

Materials and Methods

A systematic literature search of PubMed and Google Scholar was performed to identify original articles involving the AI applications in RT from the inception to 2022. Studies were included if they reported original data and explored the clinical applications of AI in RT.

Results

The selected studies were categorized into three aspects of RT: organ and lesion segmentation, treatment planning and quality assurance. For each aspect, this review discussed how these AI tools could be involved in the RT protocol.

Conclusions

Our study revealed that AI was a potential alternative for the human-dependent steps in the complex process of RT.

Historical Progress of Stereotactic Radiation Surgery

Radiosurgery and stereotactic radiotherapy have established themselves as precise and accurate areas of radiation oncology for the treatment of brain and extracranial lesions. Along with the evolution of other methods of radiotherapy, this type of treatment has been associated with significant advances in terms of a variety of modalities and techniques to improve the accuracy and efficacy of treatment. This paper provides a comprehensive overview of the progress in stereotactic radiosurgery (SRS) over several decades, and includes a review of various articles and research papers, commencing with the emergence of stereotactic techniques in radiotherapy. Key clinical aspects of SRS, such as fixation methods, radiobiology considerations, quality assurance practices, and treatment planning strategies, are presented. In addition, the review highlights the technological advancements in treatment modalities, encompassing the transition from cobalt-based systems to linear accelerator-based modalities. By addressing these topics, this study aims to offer insights into the advancements that have shaped the field of SRS, that have ultimately enhanced the accuracy and effectiveness of treatment.

EviGUIDE - a tool for evidence-based decision making in image-guided adaptive brachytherapy for cervical cancer

PURPOSE

To develop a novel decision-support system for radiation oncology that incorporates clinical, treatment and outcome data, as well as outcome models from a large clinical trial on magnetic resonance image-guided adaptive brachytherapy (MR-IGABT) for locally advanced cervical cancer (LACC).

METHODS

A system, called EviGUIDE, was developed that combines dosimetric information from the treatment planning system, patient and treatment characteristics, and established tumor control probability (TCP), and normal tissue complication probability (NTCP) models, to predict clinical outcome of radiotherapy treatment of LACC. Six Cox Proportional Hazards models based on data from 1341 patients of the EMBRACE-I study have been integrated. One TCP model for local tumor control, and five NTCP models for OAR morbidities.

RESULTS

EviGUIDE incorporates TCP-NTCP graphs to help users visualize the clinical impact of different treatment plans and provides feedback on achievable doses based on a large reference population. It enables holistic assessment of the interplay between multiple clinical endpoints and tumour and treatment variables. Retrospective analysis of 45 patients treated with MR-IGABT showed that there exists a sub-cohort of patients (20%) with increased risk factors, that could greatly benefit from the quantitative and visual feedback.

CONCLUSION

A novel digital concept was developed that can enhance clinical decision- making and facilitate personalized treatment. It serves as a proof of concept for a new generation of decision support systems in radiation oncology, which incorporate outcome models and high-quality reference data, and aids the dissemination of evidence-based knowledge about optimal treatment and serve as a blueprint for other sites in radiation oncology.

Prediction of toxicity outcomes following radiotherapy using deep learning-based models: A systematic review

PURPOSE

This study aims to perform a comprehensive systematic review of deep learning (DL) models in predicting RT-induced toxicity.

MATERIALS AND METHODS

A literature review was performed following the PRISMA guidelines. Systematic searches were performed in PubMed, Scopus, Cochrane and Embase databases from the earliest record up to September 2022. Related studies on deep learning models for radiotherapy toxicity prediction were selected based on predefined PICOS criteria.

RESULTS

Fourteen studies of radiotherapy-treated patients on different types of cancer [prostate (n=2), HNC (n=4), liver (n=2), lung (n=4), cervical (n=1), and oesophagus (n=1)] were eligible for inclusion in the systematic review. Information regarding patient characteristics and model development was summarized. Several approaches, such as ensemble learning, data augmentation, and transfer learning, that were utilized by selected studies were discussed.

CONCLUSION

Deep learning techniques are able to produce a consistent performance for toxicity prediction. Future research using large and diverse datasets and standardization of the study methodologies are required to improve the consistency of the research output.

An overview of artificial intelligence in medical physics and radiation oncology

Artificial intelligence (AI) is developing rapidly and has found widespread applications in medicine, especially radiotherapy. This paper provides a brief overview of AI applications in radiotherapy, and highlights the research directions of AI that can potentially make significant impacts and relevant ongoing research works in these directions. Challenging issues related to the clinical applications of AI, such as robustness and interpretability of AI models, are also discussed. The future research directions of AI in the field of medical physics and radiotherapy are highlighted.

Feature Importance Analysis of a Deep Learning Model for Predicting Late Bladder Toxicity Occurrence in Uterine Cervical Cancer Patients

(1) In this study, we developed a deep learning (DL) model that can be used to predict late bladder toxicity. (2) We collected data obtained from 281 uterine cervical cancer patients who underwent definitive radiation therapy. The DL model was trained using 16 features, including patient, tumor, treatment, and dose parameters, and its performance was compared with that of a multivariable logistic regression model using the following metrics: accuracy, prediction, recall, F1-score, and area under the receiver operating characteristic curve (AUROC). In addition, permutation feature importance was calculated to interpret the DL model for each feature, and the lightweight DL model was designed to focus on the top five important features. (3) The DL model outperformed the multivariable logistic regression model on our dataset. It achieved an F1-score of 0.76 and an AUROC of 0.81, while the corresponding values for the multivariable logistic regression were 0.14 and 0.43, respectively. The DL model identified the doses for the most exposed 2 cc volume of the bladder (BD_2cc) as the most important feature, followed by BD_5cc and the ICRU bladder point. In the case of the lightweight DL model, the F-score and AUROC were 0.90 and 0.91, respectively. (4) The DL models exhibited superior performance in predicting late bladder toxicity compared with the statistical method. Through the interpretation of the model, it further emphasized its potential for improving patient outcomes and minimizing treatment-related complications with a high level of reliability.

Comparison of Machine-Learning and Deep-Learning Methods for the Prediction of Osteoradionecrosis Resulting From Head and Neck Cancer Radiation Therapy

Purpose

Deep-learning (DL) techniques have been successful in disease-prediction tasks and could improve the prediction of mandible osteoradionecrosis (ORN) resulting from head and neck cancer (HNC) radiation therapy. In this study, we retrospectively compared the performance of DL algorithms and traditional machine-learning (ML) techniques to predict mandible ORN binary outcome in an extensive cohort of patients with HNC.

Methods and Materials

Patients who received HNC radiation therapy at the University of Texas MD Anderson Cancer Center from 2005 to 2015 were identified for the ML (n = 1259) and DL (n = 1236) studies. The subjects were followed for ORN development for at least 12 months, with 173 developing ORN and 1086 having no evidence of ORN. The ML models used dose-volume histogram parameters to predict ORN development. These models included logistic regression, random forest, support vector machine, and a random classifier reference. The DL models were based on ResNet, DenseNet, and autoencoder-based architectures. The DL models used each participant's dose cropped to the mandible. The effect of increasing the amount of available training data on the DL models' prediction performance was evaluated by training the DL models using increasing ratios of the original training data.

Results

The F1 score for the logistic regression model, the best-performing ML model, was 0.3. The best-performing ResNet, DenseNet, and autoencoder-based models had F1 scores of 0.07, 0.14, and 0.23, respectively, whereas the random classifier's F1 score was 0.17. No performance increase was apparent when we increased the amount of training data available for DL model training.

Conclusions

The ML models had superior performance to their DL counterparts. The lack of improvement in DL performance with increased training data suggests that either more data are needed for appropriate DL model construction or that the image features used in DL models are not suitable for this task.

Dosiomics-Based Prediction of Radiation-Induced Valvulopathy after Childhood Cancer

Valvular Heart Disease (VHD) is a known late complication of radiotherapy for childhood cancer (CC), and identifying high-risk survivors correctly remains a challenge. This paper focuses on the distribution of the radiation dose absorbed by heart tissues. We propose that a dosiomics signature could provide insight into the spatial characteristics of the heart dose associated with a VHD, beyond the already-established risk induced by high doses. We analyzed data from the 7670 survivors of the French Childhood Cancer Survivors' Study (FCCSS), 3902 of whom were treated with radiotherapy. In all, 63 (1.6%) survivors that had been treated with radiotherapy experienced a VHD, and 57 of them had heterogeneous heart doses. From the heart-dose distribution of each survivor, we extracted 93 first-order and spatial dosiomics features. We trained random forest algorithms adapted for imbalanced classification and evaluated their predictive performance compared to the performance of standard mean heart dose (MHD)-based models. Sensitivity analyses were also conducted for sub-populations of survivors with spatially heterogeneous heart doses. Our results suggest that MHD and dosiomics-based models performed equally well globally in our cohort and that, when considering the sub-population having received a spatially heterogeneous dose distribution, the predictive capability of the models is significantly improved by the use of the dosiomics features. If these findings are further validated, the dosiomics signature may be incorporated into machine learning algorithms for radiation-induced VHD risk assessment and, in turn, into the personalized refinement of follow-up guidelines.

Enhancing Radiotherapy Workflow for Head and Neck Cancer with Artificial Intelligence: A Systematic Review

BACKGROUND

Head and neck cancer (HNC) is characterized by complex-shaped tumors and numerous organs at risk (OARs), inducing challenging radiotherapy (RT) planning, optimization, and delivery. In this review, we provided a thorough description of the applications of artificial intelligence (AI) tools in the HNC RT process.

METHODS

The PubMed database was queried, and a total of 168 articles (2016-2022) were screened by a group of experts in radiation oncology. The group selected 62 articles, which were subdivided into three categories, representing the whole RT workflow: (i) target and OAR contouring, (ii) planning, and (iii) delivery.

RESULTS

The majority of the selected studies focused on the OARs segmentation process. Overall, the performance of AI models was evaluated using standard metrics, while limited research was found on how the introduction of AI could impact clinical outcomes. Additionally, papers usually lacked information about the confidence level associated with the predictions made by the AI models.

CONCLUSIONS

AI represents a promising tool to automate the RT workflow for the complex field of HNC treatment. To ensure that the development of AI technologies in RT is effectively aligned with clinical needs, we suggest conducting future studies within interdisciplinary groups, including clinicians and computer scientists.

Roadmap for precision preclinical x-ray radiation studies

This Roadmap paper covers the field of precision preclinical x-ray radiation studies in animal models. It is mostly focused on models for cancer and normal tissue response to radiation, but also discusses other disease models. The recent technological evolution in imaging, irradiation, dosimetry and monitoring that have empowered these kinds of studies is discussed, and many developments in the near future are outlined. Finally, clinical translation and reverse translation are discussed.

Toxicity Prediction in Pelvic Radiotherapy Using Multiple Instance Learning and Cascaded Attention Layers

Modern radiotherapy delivers treatment plans optimised on an individual patient level, using CT-based 3D models of patient anatomy. This optimisation is fundamentally based on simple assumptions about the relationship between radiation dose delivered to the cancer (increased dose will increase cancer control) and normal tissue (increased dose will increase rate of side effects). The details of these relationships are still not well understood, especially for radiation-induced toxicity. We propose a convolutional neural network based on multiple instance learning to analyse toxicity relationships for patients receiving pelvic radiotherapy. A dataset comprising of 315 patients were included in this study; with 3D dose distributions, pre-treatment CT scans with annotated abdominal structures, and patient-reported toxicity scores provided for each participant. In addition, we propose a novel mechanism for segregating the attentions over space and dose/imaging features independently for a better understanding of the anatomical distribution of toxicity. Quantitative and qualitative experiments were performed to evaluate the network performance. The proposed network could predict toxicity with 80% accuracy. Attention analysis over space demonstrated that there was a significant association between radiation dose to the anterior and right iliac of the abdomen and patient-reported toxicity. Experimental results showed that the proposed network had outstanding performance for toxicity prediction, localisation and explanation with the ability of generalisation for an unseen dataset.

A meta-learning approach to improving radiation response prediction in cancers

PURPOSE

Predicting the efficacy of radiotherapy in individual patients has drawn widespread attention, but the limited sample size remains a bottleneck for utilizing high-dimensional multi-omics data to guide personalized radiotherapy. We hypothesize the recently developed meta-learning framework could address this limitation.

METHODS AND MATERIALS

By combining gene expression, DNA methylation, and clinical data of 806 patients who had received radiotherapy from The Cancer Genome Atlas (TCGA), we applied the Model-Agnostic Meta-Learning (MAML) framework to tasks consisting of pan-cancer data, to obtain the best initial parameters of a neural network for a specific cancer with smaller number of samples. The performance of meta-learning framework was compared with four traditional machine learning methods based on two training schemes, and tested on Cancer Cell Line Encyclopedia (CCLE) and Chinese Glioma Genome Atlas (CGGA) datasets. Moreover, biological significance of the models was investigated by survival analysis and feature interpretation.

RESULTS

The mean AUC (Area under the ROC Curve) [95% confidence interval] of our models across nine cancer types was 0.702 [0.691-0.713], which improved by 0.166 on average over other the four machine learning methods on two training schemes. Our models performed significantly better (p < 0.05) in seven cancer types and performed comparable to the other predictors in the rest of two cancer types. The more pan-cancer samples were used to transfer meta-knowledge, the greater the performance improved (p < 0.05). The predicted response scores that our models generated were negatively correlated with cell radiosensitivity index in four cancer types (p < 0.05), while not statistically significant in the other three cancer types. Moreover, the predicted response scores were shown to be prognostic factors in seven cancer types and eight potential radiosensitivity-related genes were identified.

CONCLUSIONS

For the first time, we established the meta-learning approach to improving individual radiation response prediction by transferring common knowledge from pan-cancer data with MAML framework. The results demonstrated the superiority, generalizability, and biological significance of our approach.

Current status and future developments in predicting outcomes in radiation oncology

Advancements in data-driven technologies and the inclusion of information-rich multiomics features have significantly improved the performance of outcomes modeling in radiation oncology. For this current trend to be sustainable, challenges related to robust data modeling such as small sample size, low size to feature ratio, noisy data, as well as issues related to algorithmic modeling such as complexity, uncertainty, and interpretability, need to be mitigated if not resolved. Emerging computational technologies and new paradigms such as federated learning, human-in-the-loop, quantum computing, and novel interpretability methods show great potential in overcoming these challenges and bridging the gap towards precision outcome modeling in radiotherapy. Examples of these promising technologies will be presented and their potential role in improving outcome modeling will be discussed.

Artificial intelligence in radiotherapy

Radiotherapy is a discipline closely integrated with computer science. Artificial intelligence (AI) has developed rapidly over the past few years. With the explosive growth of medical big data, AI promises to revolutionize the field of radiotherapy through highly automated workflow, enhanced quality assurance, improved regional balances of expert experiences, and individualized treatment guided by multi-omics. In addition to independent researchers, the increasing number of large databases, biobanks, and open challenges significantly facilitated AI studies on radiation oncology. This article reviews the latest research, clinical applications, and challenges of AI in each part of radiotherapy including image processing, contouring, planning, quality assurance, motion management, and outcome prediction. By summarizing cutting-edge findings and challenges, we aim to inspire researchers to explore more future possibilities and accelerate the arrival of AI radiotherapy.

18F-FDG PET Radiomics as Predictor of Treatment Response in Oesophageal Cancer: A Systematic Review and Meta-Analysis

The best treatment strategy for oesophageal cancer patients achieving a complete clinical response after neoadjuvant chemoradiation is a burning topic. The available diagnostic tools, such as 18F-FDG PET/CT performed routinely, cannot accurately evaluate the presence or absence of the residual tumour. The emerging field of radiomics may encounter the critical challenge of personalised treatment. Radiomics is based on medical image analysis, executed by extracting information from many image features; it has been shown to provide valuable information for predicting treatment responses in oesophageal cancer. This systematic review with a meta-analysis aims to provide current evidence of 18F-FDG PET-based radiomics in predicting response treatments following neoadjuvant chemoradiotherapy in oesophageal cancer. A comprehensive literature review identified 1160 studies, of which five were finally included in the study. Our findings provided that pooled Area Under the Curve (AUC) of the five selected studies was relatively high at 0.821 (95% CI: 0.737–0.904) and not influenced by the sample size of the studies. Radiomics models exhibited a good performance in predicting pathological complete responses (pCRs). This review further strengthens the great potential of 18F-FDG PET-based radiomics to predict pCRs in oesophageal cancer patients who underwent neoadjuvant chemoradiotherapy. Additionally, our review imparts additional support to prospective studies on 18F-FDG PET radiomics for a tailored treatment strategy of oesophageal cancer patients.