The Application and Development of Deep Learning in Radiotherapy: A Systematic Review

On-board synthetic 4D MRI generation from 4D CBCT for radiotherapy of abdominal tumors: A feasibility study

BACKGROUND

Magnetic resonance-guided radiotherapy with an MR-guided LINAC represents potential clinical benefits in abdominal treatments due to the superior soft-tissue contrast compared to kV-based images in conventional treatment units. However, due to the high cost associated with this technology, only a few centers have access to it. As an alternative, synthetic 4D MRI generation based on artificial intelligence methods could be implemented. Nevertheless, appropriate MRI texture generation from CT images might be challenging and prone to hallucinations, compromising motion accuracy.

PURPOSE

To evaluate the feasibility of on-board synthetic motion-resolved 4D MRI generation from prior 4D MRI, on-board 4D cone beam CT (CBCT) images, motion modeling information, and deep learning models using the digital anthropomorphic phantom XCAT.

METHODS

The synthetic 4D MRI corresponds to phases from on-board 4D CBCT. Each synthetic MRI volume in the 4D MRI was generated by warping a reference 3D MRI (MRIref, end of expiration phase from a prior 4D MRI) with a deformation field map (DFM) determined by (I) the eigenvectors from the principal component analysis (PCA) motion-modeling of the prior 4D MRI, and (II) the corresponding eigenvalues predicted by a convolutional neural network (CNN) model using the on-board 4D CBCT images as input. The CNN was trained with 1000 deformations of one reference CT (CTref, same conditions as MRIref) generated by applying 1000 DFMs computed by randomly sampling the original eigenvalues from the prior 4D MRI PCA model. The evaluation metrics for the CNN model were root-mean-square error (RMSE) and mean absolute error (MAE). Finally, different on-board 4D-MRI generation scenarios were assessed by changing the respiratory period, the amplitude of the diaphragm, and the chest wall motion of the 4D CBCT using normalized root-mean-square error (nRMSE) and structural similarity index measure (SSIM) for image-based evaluation, and volume dice coefficient (VDC), volume percent difference (VPD), and center-of-mass shift (COMS) for contour-based evaluation of liver and target volumes.

RESULTS

The RMSE and MAE values of the CNN model reported 0.012 ± 0.001 and 0.010 ± 0.001, respectively for the first eigenvalue predictions. SSIM and nRMSE were 0.96 ± 0.06 and 0.22 ± 0.08, respectively. VDC, VPD, and COMS were 0.92 ± 0.06, 3.08 ± 3.73 %, and 2.3 ± 2.1 mm, respectively, for the target volume. The more challenging synthetic 4D-MRI generation scenario was for one 4D-CBCT with increased chest wall motion amplitude, reporting SSIM and nRMSE of 0.82 and 0.51, respectively.

CONCLUSIONS

On-board synthetic 4D-MRI generation based on predicting actual treatment deformation from on-board 4D-CBCT represents a method that can potentially improve the treatment-setup localization in abdominal radiotherapy treatments with a conventional kV-based LINAC.

Deep learning applied to dose prediction in external radiation therapy: A narrative review

Over the last decades, the use of artificial intelligence, machine learning and deep learning in medical fields has skyrocketed. Well known for their results in segmentation, motion management and posttreatment outcome tasks, investigations of machine learning and deep learning models as fast dose calculation or quality assurance tools have been present since 2000. The main motivation for this increasing research and interest in artificial intelligence, machine learning and deep learning is the enhancement of treatment workflows, specifically dosimetry and quality assurance accuracy and time points, which remain important time-consuming aspects of clinical patient management. Since 2014, the evolution of models and architectures for dose calculation has been related to innovations and interest in the theory of information research with pronounced improvements in architecture design. The use of knowledge-based approaches to patient-specific methods has also considerably improved the accuracy of dose predictions. This paper covers the state of all known deep learning architectures and models applied to external radiotherapy with a description of each architecture, followed by a discussion on the performance and future of deep learning predictive models in external radiotherapy.

Deep learning for autosegmentation for radiotherapy treatment planning: State-of-the-art and novel perspectives

The rapid development of artificial intelligence (AI) has gained importance, with many tools already entering our daily lives. The medical field of radiation oncology is also subject to this development, with AI entering all steps of the patient journey. In this review article, we summarize contemporary AI techniques and explore the clinical applications of AI-based automated segmentation models in radiotherapy planning, focusing on delineation of organs at risk (OARs), the gross tumor volume (GTV), and the clinical target volume (CTV). Emphasizing the need for precise and individualized plans, we review various commercial and freeware segmentation tools and also state-of-the-art approaches. Through our own findings and based on the literature, we demonstrate improved efficiency and consistency as well as time savings in different clinical scenarios. Despite challenges in clinical implementation such as domain shifts, the potential benefits for personalized treatment planning are substantial. The integration of mathematical tumor growth models and AI-based tumor detection further enhances the possibilities for refining target volumes. As advancements continue, the prospect of one-stop-shop segmentation and radiotherapy planning represents an exciting frontier in radiotherapy, potentially enabling fast treatment with enhanced precision and individualization.

Application and progress of artificial intelligence in radiation therapy dose prediction

Radiation therapy (RT) nowadays is a main treatment modality of cancer. To ensure the therapeutic efficacy of patients, accurate dose distribution is often required, which is a time-consuming and labor-intensive process. In addition, due to the differences in knowledge and experience among participants and diverse institutions, the predicted dose are often inconsistent. In last several decades, artificial intelligence (AI) has been applied in various aspects of RT, several products have been implemented in clinical practice and confirmed superiority. In this paper, we will review the research of AI in dose prediction, focusing on the progress in deep learning (DL).

Optimal timing of re-planning for head and neck adaptive radiotherapy

BACKGROUND AND PURPOSE

Adaptive radiotherapy (ART) relies on re-planning to correct treatment variations, but the optimal timing of re-planning to account for dose changes in head and neck organs at risk (OARs) is still under investigation. We aimed to find out the optimal timing of re-planning in head and neck ART.

MATERIALS AND METHODS

A total of 110 head and neck cancer patients were retrospectively enrolled. A semi auto-segmentation method was applied to obtain the weekly mean dose (Dmean) to OARs. The K-nearest-neighbour method was used for missing data imputation of weekly Dmean. A dose deviation map was built using the planning Dmean and weekly Dmean values and then used to simulate different ART scenarios consisting of 1 to 6 re-plannings. The difference between accumulated Dmean and planning Dmean before re-planning (ΔDmean_acc_noART) and after re-planning (ΔDmean_acc_ART) were evaluated and compared.

RESULTS

Among all the OARs, supraglottic showed the largest ΔDmean_acc_noART (1.23 ± 3.13 Gy) and most cases of ΔDmean_acc_noART > 3 Gy (26 patients). The 3rd week is suggested in the optimal timing of re-planning for 10 OARs. For all the organs except arytenoid, 2 re-plannings were able to guarantee the ΔDmean_acc_ART below 3 Gy while the average |ΔDmean_acc_ART| was below 1 Gy. ART scenarios of 2_4, 3_4, 3_5 (week of re-planning separated with "_") were able to guarantee ΔDmean_acc_ART of 99 % of patients below 3 Gy simultaneously for 19 OARs.

CONCLUSIONS

The optimal timing of re-planning was suggested for different organs at risk in head and neck adaptive radiotherapy. Generic scenarios of timing and frequency for re-planning can be applied to guarantee the increase of accumulated mean dose within 3 Gy simultaneously for multiple organs.

The Integration of Deep Learning in Radiotherapy: Exploring Challenges, Opportunities, and Future Directions through an Umbrella Review

This study investigates, through a narrative review, the transformative impact of deep learning (DL) in the field of radiotherapy, particularly in light of the accelerated developments prompted by the COVID-19 pandemic. The proposed approach was based on an umbrella review following a standard narrative checklist and a qualification process. The selection process identified 19 systematic review studies. Through an analysis of current research, the study highlights the revolutionary potential of DL algorithms in optimizing treatment planning, image analysis, and patient outcome prediction in radiotherapy. It underscores the necessity of further exploration into specific research areas to unlock the full capabilities of DL technology. Moreover, the study emphasizes the intricate interplay between digital radiology and radiotherapy, revealing how advancements in one field can significantly influence the other. This interdependence is crucial for addressing complex challenges and advancing the integration of cutting-edge technologies into clinical practice. Collaborative efforts among researchers, clinicians, and regulatory bodies are deemed essential to effectively navigate the evolving landscape of DL in radiotherapy. By fostering interdisciplinary collaborations and conducting thorough investigations, stakeholders can fully leverage the transformative power of DL to enhance patient care and refine therapeutic strategies. Ultimately, this promises to usher in a new era of personalized and optimized radiotherapy treatment for improved patient outcomes.

Hippocampal sparing in whole-brain radiotherapy for brain metastases: controversy, technology and the future

Whole-brain radiotherapy (WBRT) plays an irreplaceable role in the treatment of brain metastases (BMs), but cognitive decline after WBRT seriously affects patients' quality of life. The development of cognitive dysfunction is closely related to hippocampal injury, but standardized criteria for predicting hippocampal injury and dose limits for hippocampal protection have not yet been developed. This review systematically reviews the clinical efficacy of hippocampal avoidance - WBRT (HA-WBRT), the controversy over dose limits, common methods and characteristics of hippocampal imaging and segmentation, differences in hippocampal protection by common radiotherapy (RT) techniques, and the application of artificial intelligence (AI) and radiomic techniques for hippocampal protection. In the future, the application of new techniques and methods can improve the consistency of hippocampal dose limit determination and the prediction of the occurrence of cognitive dysfunction in WBRT patients, avoiding the occurrence of cognitive dysfunction in patients and thus benefiting more patients with BMs.

Clinical assessment of deep learning-based uncertainty maps in lung cancer segmentation

Objective. Prior to radiation therapy planning, accurate delineation of gross tumour volume (GTVs) and organs at risk (OARs) is crucial. In the current clinical practice, tumour delineation is performed manually by radiation oncologists, which is time-consuming and prone to large inter-observer variability. With the advent of deep learning (DL) models, automated contouring has become possible, speeding up procedures and assisting clinicians. However, these tools are currently used in the clinic mostly for contouring OARs, since these systems are not reliable yet for contouring GTVs. To improve the reliability of these systems, researchers have started exploring the topic of probabilistic neural networks. However, there is still limited knowledge of the practical implementation of such networks in real clinical settings.Approach. In this work, we developed a 3D probabilistic system that generates DL-based uncertainty maps for lung cancer CT segmentations. We employed the Monte Carlo (MC) dropout technique to generate probabilistic and uncertainty maps, while the model calibration was evaluated by using reliability diagrams. A clinical validation was conducted in collaboration with a radiation oncologist to qualitatively assess the value of the uncertainty estimates. We also proposed two novel metrics, namely mean uncertainty (MU) and relative uncertainty volume (RUV), as potential indicators for clinicians to assess the need for independent visual checks of the DL-based segmentation. Main results. Our study showed that uncertainty mapping effectively identified cases of under or over-contouring. Although the overconfidence of the model, a strong correlation was observed between the clinical opinion and MU metric. Moreover, both MU and RUV revealed high AUC values in discretising between low and high uncertainty cases.Significance. Our study is one of the first attempts to clinically validate uncertainty estimates in DL-based contouring. The two proposed metrics exhibited promising potential as indicators for clinicians to independently assess the quality of tumour delineation.

A clinical evaluation of the performance of five commercial artificial intelligence contouring systems for radiotherapy

Purpose/objectives

Auto-segmentation with artificial intelligence (AI) offers an opportunity to reduce inter- and intra-observer variability in contouring, to improve the quality of contours, as well as to reduce the time taken to conduct this manual task. In this work we benchmark the AI auto-segmentation contours produced by five commercial vendors against a common dataset.

Methods and materials

The organ at risk (OAR) contours generated by five commercial AI auto-segmentation solutions (Mirada (Mir), MVision (MV), Radformation (Rad), RayStation (Ray) and TheraPanacea (Ther)) were compared to manually-drawn expert contours from 20 breast, 20 head and neck, 20 lung and 20 prostate patients. Comparisons were made using geometric similarity metrics including volumetric and surface Dice similarity coefficient (vDSC and sDSC), Hausdorff distance (HD) and Added Path Length (APL). To assess the time saved, the time taken to manually draw the expert contours, as well as the time to correct the AI contours, were recorded.

Results

There are differences in the number of CT contours offered by each AI auto-segmentation solution at the time of the study (Mir 99; MV 143; Rad 83; Ray 67; Ther 86), with all offering contours of some lymph node levels as well as OARs. Averaged across all structures, the median vDSCs were good for all systems and compared favorably with existing literature: Mir 0.82; MV 0.88; Rad 0.86; Ray 0.87; Ther 0.88. All systems offer substantial time savings, ranging between: breast 14-20 mins; head and neck 74-93 mins; lung 20-26 mins; prostate 35-42 mins. The time saved, averaged across all structures, was similar for all systems: Mir 39.8 mins; MV 43.6 mins; Rad 36.6 min; Ray 43.2 mins; Ther 45.2 mins.

Conclusions

All five commercial AI auto-segmentation solutions evaluated in this work offer high quality contours in significantly reduced time compared to manual contouring, and could be used to render the radiotherapy workflow more efficient and standardized.

Effects of Patchwise Sampling Strategy to Three-Dimensional Convolutional Neural Network-Based Alzheimer's Disease Classification

In recent years, the rapid development of artificial intelligence has promoted the widespread application of convolutional neural networks (CNNs) in neuroimaging analysis. Although three-dimensional (3D) CNNs can utilize the spatial information in 3D volumes, there are still some challenges related to high-dimensional features and potential overfitting issues. To overcome these problems, patch-based CNNs have been used, which are beneficial for model generalization. However, it is unclear how the choice of a patchwise sampling strategy affects the performance of the Alzheimer's Disease (AD) classification. To this end, the present work investigates the impact of a patchwise sampling strategy for 3D CNN based AD classification. A 3D framework cascaded by two-stage subnetworks was used for AD classification. The patch-level subnetworks learned feature representations from local image patches, and the subject-level subnetwork combined discriminative feature representations from all patch-level subnetworks to generate a classification score at the subject level. Experiments were conducted to determine the effect of patch partitioning methods, the effect of patch size, and interactions between patch size and training set size for AD classification. With the same data size and identical network structure, the 3D CNN model trained with 48 × 48 × 48 cubic image patches showed the best performance in AD classification (ACC = 89.6%). The model trained with hippocampus-centered, region of interest (ROI)-based image patches showed suboptimal performance. If the pathological features are concentrated only in some regions affected by the disease, the empirically predefined ROI patches might be the right choice. The better performance of cubic image patches compared with cuboidal image patches is likely related to the pathological distribution of AD. The image patch size and training sample size together have a complex influence on the performance of the classification. The size of the image patches should be determined based on the size of the training sample to compensate for noisy labels and the problem of the curse of dimensionality. The conclusions of the present study can serve as a reference for the researchers who wish to develop a superior 3D patch-based CNN model with an appropriate patch sampling strategy.

A Comprehensive Survey on Deep-Learning-Based Breast Cancer Diagnosis

Breast cancer is now the most frequently diagnosed cancer in women, and its percentage is gradually increasing. Optimistically, there is a good chance of recovery from breast cancer if identified and treated at an early stage. Therefore, several researchers have established deep-learning-based automated methods for their efficiency and accuracy in predicting the growth of cancer cells utilizing medical imaging modalities. As of yet, few review studies on breast cancer diagnosis are available that summarize some existing studies. However, these studies were unable to address emerging architectures and modalities in breast cancer diagnosis. This review focuses on the evolving architectures of deep learning for breast cancer detection. In what follows, this survey presents existing deep-learning-based architectures, analyzes the strengths and limitations of the existing studies, examines the used datasets, and reviews image pre-processing techniques. Furthermore, a concrete review of diverse imaging modalities, performance metrics and results, challenges, and research directions for future researchers is presented.