An Interpretable Planning Bot for Pancreas Stereotactic Body Radiation Therapy

Deep reinforcement learning in radiation therapy planning optimization: A comprehensive review

PURPOSE

The formulation and optimization of radiation therapy plans are complex and time-consuming processes that heavily rely on the expertise of medical physicists. Consequently, there is an urgent need for automated optimization methods. Recent advancements in reinforcement learning, particularly deep reinforcement learning (DRL), show great promise for automating radiotherapy planning. This review summarizes the current state of DRL applications in this field, evaluates their effectiveness, and identifies challenges and future directions.

METHODS

A systematic search was conducted in Google Scholar, PubMed, IEEE Xplore, and Scopus using keywords such as "deep reinforcement learning", "radiation therapy", and "treatment planning". The extracted data were synthesized for an overview and critical analysis.

RESULTS

The application of deep reinforcement learning in radiation therapy plan optimization can generally be divided into three categories: optimizing treatment planning parameters, directly optimizing machine parameters, and adaptive radiotherapy. From the perspective of disease sites, DRL has been applied to cervical cancer, prostate cancer, vestibular schwannoma, and lung cancer. Regarding types of radiation therapy, it has been used in HDRBT, IMRT, SBRT, VMAT, GK, and Cyberknife.

CONCLUSIONS

Deep reinforcement learning technology has played a significant role in advancing the automated optimization of radiation therapy plans. However, there is still a considerable gap before it can be widely applied in clinical settings due to three main reasons: inefficiency, limited methods for quality assessment, and poor interpretability. To address these challenges, significant research opportunities exist in the future, such as constructing evaluators, parallelized training, and exploring continuous action spaces.

Artificial intelligence in radiotherapy: Current applications and future trends

Radiation therapy has dramatically changed with the advent of computed tomography and intensity modulation. This added complexity to the workflow but allowed for more precise and reproducible treatment. As a result, these advances required the accurate delineation of many more volumes, raising questions about how to delineate them, in a uniform manner across centers. Then, as computing power improved, reverse planning became possible and three-dimensional dose distributions could be generated. Artificial intelligence offers the opportunity to make such workflow more efficient while increasing practice homogeneity. Many artificial intelligence-based tools are being implemented in routine practice to increase efficiency, reduce workload and improve homogeneity of treatments. Data retrieved from this workflow could be combined with clinical data and omic data to develop predictive tools to support clinical decision-making process. Such predictive tools are at the stage of proof-of-concept and need to be explainatory, prospectively validated, and based on large and multicenter cohorts. Nevertheless, they could bridge the gap to personalized radiation oncology, by personalizing oncologic strategies, dose prescriptions to tumor volumes and dose constraints to organs at risk.

Human-like intelligent automatic treatment planning of head and neck cancer radiation therapy

Objective.Automatic treatment planning of radiation therapy (RT) is desired to ensure plan quality, improve planning efficiency, and reduce human errors. We have proposed an Intelligent Automatic Treatment Planning framework with a virtual treatment planner (VTP), an artificial intelligence robot built using deep reinforcement learning, autonomously operating a treatment planning system (TPS). This study extends our previous successes in relatively simple prostate cancer RT planning to head-and-neck (H&N) cancer, a more challenging context even for human planners due to multiple prescription levels, proximity of targets to critical organs, and tight dosimetric constraints.Approach.We integrated VTP with a real clinical TPS to establish a fully automated planning workflow guided by VTP. This integration allowed direct model training and evaluation using the clinical TPS. We designed the VTP network structure to approach the decision-making process in RT planning in a hierarchical manner that mirrors human planners. The VTP network was trained via theQ-learning framework. To assess the effectiveness of VTP, we conducted a prospective evaluation in the 2023 Planning Challenge organized by the American Association of Medical Dosimetrists (AAMD). We extended our evaluation to include 20 clinical H&N cancer patients, comparing the plans generated by VTP against their clinical plans.Main results.In the prospective evaluation for the AAMD Planning Challenge, VTP achieved a plan score of 139.08 in the initial phase evaluating plan quality, and 15 min of planning time with the first place ranking in the adaptive phase competing for planning efficiency while meeting all plan quality requirements. For clinical cases, VTP-generated plans achieved an average VTP score of125.33±11.12, which outperformed the corresponding clinical plans with an average score of117.76±13.56.Significance.We successfully integrated VTP with the clinical TPS to achieve a fully automated treatment planning workflow. The compelling performance of VTP demonstrated its potential in automating H&N cancer RT planning.

A retrospective study on the investigation of potential dosimetric benefits of online adaptive proton therapy for head and neck cancer

PURPOSE

Proton therapy is sensitive to anatomical changes, often occurring in head-and-neck (HN) cancer patients. Although multiple studies have proposed online adaptive proton therapy (APT), there is still a concern in the radiotherapy community about the necessity of online APT. We have performed a retrospective study to investigate the potential dosimetric benefits of online APT for HN patients relative to the current offline APT.

METHODS

Our retrospective study has a patient cohort of 10 cases. To mimic online APT, we re-evaluated the dose of the in-use treatment plan on patients' actual treatment anatomy captured by cone-beam CT (CBCT) for each fraction and performed a templated-based automatic replanning if needed, assuming that these were performed online before treatment delivery. Cumulative dose of the simulated online APT course was calculated and compared with that of the actual offline APT course and the designed plan dose of the initial treatment plan (referred to as nominal plan). The ProKnow scoring system was employed and adapted for our study to quantify the actual quality of both courses against our planning goals.

RESULTS

The average score of the nominal plans over the 10 cases is 41.0, while those of the actual offline APT course and our simulated online course is 25.8 and 37.5, respectively. Compared to the offline APT course, our online course improved dose quality for all cases, with the score improvement ranging from 0.4 to 26.9 and an average improvement of 11.7.

CONCLUSION

The results of our retrospective study have demonstrated that online APT can better address anatomical changes for HN cancer patients than the current offline replanning practice. The advanced artificial intelligence based automatic replanning technology presents a promising avenue for extending potential benefits of online APT.

Predicting treatment plan approval probability for high-dose-rate brachytherapy of cervical cancer using adversarial deep learning

Objective.Predicting the probability of having the plan approved by the physician is important for automatic treatment planning. Driven by the mathematical foundation of deep learning that can use a deep neural network to represent functions accurately and flexibly, we developed a deep-learning framework that learns the probability of plan approval for cervical cancer high-dose-rate brachytherapy (HDRBT).Approach.The system consisted of a dose prediction network (DPN) and a plan-approval probability network (PPN). DPN predicts organs at risk (OAR)D2ccand CTVD90%of the current fraction from the patient's current anatomy and prescription dose of HDRBT. PPN outputs the probability of a given plan being acceptable to the physician based on the patients anatomy and the total dose combining HDRBT and external beam radiotherapy sessions. Training of the networks was achieved by first training them separately for a good initialization, and then jointly via an adversarial process. We collected approved treatment plans of 248 treatment fractions from 63 patients. Among them, 216 plans from 54 patients were employed in a four-fold cross validation study, and the remaining 32 plans from other 9 patients were saved for independent testing.Main results.DPN predicted equivalent dose of 2 Gy for bladder, rectum, sigmoidD2ccand CTVD90%with a relative error of 11.51% ± 6.92%, 8.23% ± 5.75%, 7.12% ± 6.00%, and 10.16% ± 10.42%, respectively. In a task that differentiates clinically approved plans and disapproved plans generated by perturbing doses in ground truth approved plans by 20%, PPN achieved accuracy, sensitivity, specificity, and area under the curve 0.70, 0.74, 0.65, and 0.74.Significance.We demonstrated the feasibility of developing a novel deep-learning framework that predicts a probability of plan approval for HDRBT of cervical cancer, which is an essential component in automatic treatment planning.

A Review of Artificial Intelligence Application for Radiotherapy

Background and Purpose

Artificial intelligence (AI) is a technique which tries to think like humans and mimic human behaviors. It has been considered as an alternative in a lot of human-dependent steps in radiotherapy (RT), since the human participation is a principal uncertainty source in RT. The aim of this work is to provide a systematic summary of the current literature on AI application for RT, and to clarify its role for RT practice in terms of clinical views.

Materials and Methods

A systematic literature search of PubMed and Google Scholar was performed to identify original articles involving the AI applications in RT from the inception to 2022. Studies were included if they reported original data and explored the clinical applications of AI in RT.

Results

The selected studies were categorized into three aspects of RT: organ and lesion segmentation, treatment planning and quality assurance. For each aspect, this review discussed how these AI tools could be involved in the RT protocol.

Conclusions

Our study revealed that AI was a potential alternative for the human-dependent steps in the complex process of RT.

"Under the hood": artificial intelligence in personalized radiotherapy

This review presents and discusses the ways in which artificial intelligence (AI) tools currently intervene, or could potentially intervene in the future, to enhance the diverse tasks involved in the radiotherapy workflow. The radiotherapy framework is presented on 2 different levels for the personalization of the treatment, distinct in tasks and methodologies. The first level is the clinically well-established anatomy-based workflow, known as adaptive radiation therapy. The second level is referred to as biology-driven workflow, explored in the research literature and recently appearing in some preliminary clinical trials for personalized radiation treatments. A 2-fold role for AI is defined according to these 2 different levels. In the anatomy-based workflow, the role of AI is to streamline and improve the tasks in terms of time and variability reductions compared to conventional methodologies. The biology-driven workflow instead fully relies on AI, which introduces decision-making tools opening uncharted frontiers that were in the past deemed challenging to explore. These methodologies are referred to as radiomics and dosiomics, handling imaging and dosimetric information, or multiomics, when complemented by clinical and biological parameters (ie, biomarkers). The review explicitly highlights the methodologies that are currently incorporated into clinical practice or still in research, with the aim of presenting the AI's growing role in personalized radiotherapy.

Interpretable artificial intelligence in radiology and radiation oncology

Artificial intelligence has been introduced to clinical practice, especially radiology and radiation oncology, from image segmentation, diagnosis, treatment planning and prognosis. It is not only crucial to have an accurate artificial intelligence model, but also to understand the internal logic and gain the trust of the experts. This review is intended to provide some insights into core concepts of the interpretability, the state-of-the-art methods for understanding the machine learning models, the evaluation of these methods, identifying some challenges and limits of them, and gives some examples of medical applications.

Implementation and evaluation of an intelligent automatic treatment planning robot for prostate cancer stereotactic body radiation therapy

PURPOSE

We previously developed a virtual treatment planner (VTP), an artificial intelligence robot, operating a treatment planning system (TPS). Using deep reinforcement learning guided by human knowledge, we trained the VTP to autonomously adjust relevant parameters in treatment plan optimization, similar to a human planner, to generate high-quality plans for prostate cancer stereotactic body radiation therapy (SBRT). This study describes the clinical implementation and evaluation of VTP.

MATERIALS AND METHODS

We integrate VTP with Eclipse TPS using scripting Application Programming Interface. VTP observes dose-volume histograms of relevant structures, decides how to adjust dosimetric constraints, including doses, volumes, and weighting factors, and applies the adjustments to the TPS interface to launch the optimization engine. This process continues until a high-quality plan is achieved. We evaluated VTP's performance using the prostate SBRT case from the 2016 American Association of Medical Dosimetrist/Radiosurgery Society plan study with its plan scoring system, and compared to human-generated plans submitted to the challenge. Using the same scoring system, we also compared the plan quality of 36 prostate SBRT cases (20 planned with IMRT and 16 planned with VMAT) treated at our institution for both VTP and human-generated plans.

RESULTS

In the plan study case, VTP achieved a score of 142.1/150.0, ranking the third in the competition (median 134.6). For the clinical cases, VTP achieved 110.6 ± 6.5 for 20 IMRT plans and 126.2 ± 4.7 for 16 VMAT plans, similar to scores of human-generated plans with 110.4 ± 7.0 for IMRT plans and 125.4 ± 4.4 for VMAT plans. The workflow, plan quality and planning time of VTP were reviewed to be satisfactory by experienced physicists.

CONCLUSION

We successfully implemented VTP to operate a TPS for autonomous human-like treatment planning for prostate SBRT.

Reinforcement learning strategies in cancer chemotherapy treatments: A review

BACKGROUND AND OBJECTIVE

Cancer is one of the major causes of death worldwide and chemotherapies are the most significant anti-cancer therapy, in spite of the emerging precision cancer medicines in the last 2 decades. The growing interest in developing the effective chemotherapy regimen with optimal drug dosing schedule to benefit the clinical cancer patients has spawned innovative solutions involving mathematical modeling since the chemotherapy regimens are administered cyclically until the futility or the occurrence of intolerable adverse events. Thus, in this present work, we reviewed the emerging trends involved in forming a computational solution from the aspect of reinforcement learning.

METHODS

Initially, this survey in-depth focused on the details of the dynamic treatment regimens from a broad perspective and then narrowed down to inspirations from reinforcement learning that were advantageous to chemotherapy dosing, including both offline reinforcement learning and supervised reinforcement learning.

RESULTS

The insights established in the chemotherapy-planning problem associated with the Reinforcement Learning (RL) has been discussed in this study. It showed that the researchers were able to widen their perspectives in comprehending the theoretical basis, dynamic treatment regimens (DTR), use of the adaptive control on DTR, and the associated RL techniques.

CONCLUSIONS

This study reviewed the recent researches relevant to the topic, and highlighted the challenges, open questions, possible solutions, and future steps in inventing a realistic solution for the aforementioned problem.

Evaluating the plan quality of a general head-and-neck knowledge-based planning model versus separate unilateral/bilateral models

The implementation of knowledge-based planning (KBP) continues to grow in radiotherapy clinics. KBP guides radiation treatment design by generating clinically acceptable plans in a timely and resource-efficient manner. The role of multiple KBP models tailored for variations within a disease site remains undefined in part because of the substantial effort and number of training cases required to create a high-quality KBP model. In this study, our aim was to explore whether site-specific KBP models lead to clinically meaningful differences in plan quality for head-and-neck (HN) patients when compared to a general model. One KBP model was created from prior volumetric-modulated arc therapy (VMAT) cases that treated unilateral HN lymph nodes while another model was created from VMAT cases that treated bilateral HN nodes. Thirty cases from each model (60 cases total) were randomly selected to create a third, general model. These models were applied to 60 HN test cases - 30 unilateral and 30 bilateral - to generate 180 VMAT plans in Eclipse. Clinically relevant dose metrics were compared between models. Paired-sample t-tests were used for statistical analysis, with the threshold for statistical significance set a priori at 0.007, taking into consideration multiple hypothesis testing to avoid type I error. For unilateral test cases, the unilateral model-generated plans had significantly lower spinal cord maximum doses (12.1 Gy vs 19.3 Gy, p < 0.001) and oral cavity mean doses (20.8 Gy vs 23.0 Gy, p < 0.001), compared with the bilateral model-generated plans. The unilateral and general models generated comparable plans for unilateral HN test cases. For bilateral test cases, the bilateral model created plans had significantly lower brainstem maximum doses (10.8 Gy vs 12.2 Gy, p < 0.001) and parotid mean doses (24.0 Gy vs 25.5 Gy, p < 0.001) when compared to the unilateral model. Right parotid mean doses were lower for bilateral model plans compared to general model plans (23.8 Gy vs 24.4 Gy). The general model created plans with significantly lower brainstem maximum doses (10.3 Gy vs 10.8 Gy) and oral cavity mean doses (35.3 Gy vs 36.7 Gy) when compared with bilateral model-generated plans. The general model outperformed the bilateral model in several dose metrics but they were not deemed clinically significant. For both case sets, the unilateral and general model created plans had higher monitor units when compared to the bilateral model, likely due to more stringent constraint settings. All other dose metrics were comparable. This study demonstrates that a balanced general HN model created using carefully curated treatment plans can produce high quality plans comparable to dedicated unilateral and bilateral models.

Deep reinforcement learning and its applications in medical imaging and radiation therapy: a survey

Reinforcement learning takes sequential decision-making approaches by learning the policy through trial and error based on interaction with the environment. Combining deep learning and reinforcement learning can empower the agent to learn the interactions and the distribution of rewards from state-action pairs to achieve effective and efficient solutions in more complex and dynamic environments. Deep reinforcement learning (DRL) has demonstrated astonishing performance in surpassing the human-level performance in the game domain and many other simulated environments. This paper introduces the basics of reinforcement learning and reviews various categories of DRL algorithms and DRL models developed for medical image analysis and radiation treatment planning optimization. We will also discuss the current challenges of DRL and approaches proposed to make DRL more generalizable and robust in a real-world environment. DRL algorithms, by fostering the designs of the reward function, agents interactions and environment models, can resolve the challenges from scarce and heterogeneous annotated medical image data, which has been a major obstacle to implementing deep learning models in the clinic. DRL is an active research area with enormous potential to improve deep learning applications in medical imaging and radiation therapy planning.

Artificial intelligence in pancreatic cancer

Pancreatic cancer is the deadliest disease, with a five-year overall survival rate of just 11%. The pancreatic cancer patients diagnosed with early screening have a median overall survival of nearly ten years, compared with 1.5 years for those not diagnosed with early screening. Therefore, early diagnosis and early treatment of pancreatic cancer are particularly critical. However, as a rare disease, the general screening cost of pancreatic cancer is high, the accuracy of existing tumor markers is not enough, and the efficacy of treatment methods is not exact. In terms of early diagnosis, artificial intelligence technology can quickly locate high-risk groups through medical images, pathological examination, biomarkers, and other aspects, then screening pancreatic cancer lesions early. At the same time, the artificial intelligence algorithm can also be used to predict the survival time, recurrence risk, metastasis, and therapy response which could affect the prognosis. In addition, artificial intelligence is widely used in pancreatic cancer health records, estimating medical imaging parameters, developing computer-aided diagnosis systems, etc. Advances in AI applications for pancreatic cancer will require a concerted effort among clinicians, basic scientists, statisticians, and engineers. Although it has some limitations, it will play an essential role in overcoming pancreatic cancer in the foreseeable future due to its mighty computing power.

The development of a deep reinforcement learning network for dose-volume-constrained treatment planning in prostate cancer intensity modulated radiotherapy

Although commercial treatment planning systems (TPSs) can automatically solve the optimization problem for treatment planning, human planners need to define and adjust the planning objectives/constraints to obtain clinically acceptable plans. Such a process is labor-intensive and time-consuming. In this work, we show an end-to-end study to train a deep reinforcement learning (DRL) based virtual treatment planner (VTP) that can behave like a human to operate a dose-volume constrained treatment plan optimization engine following the parameters used in Eclipse TPS for high-quality treatment planning. We considered the prostate cancer IMRT treatment plan as the testbed. The VTP took the dose-volume histogram (DVH) of a plan as input and predicted the optimal strategy for constraint adjustment to improve the plan quality. The training of VTP followed the state-of-the-art Q-learning framework. Experience replay was implemented with epsilon-greedy search to explore the impacts of taking different actions on a large number of automatically generated plans, from which an optimal policy can be learned. Since a major computational cost in training was to solve the plan optimization problem repeatedly, we implemented a graphical processing unit (GPU)-based technique to improve the efficiency by 2-fold. Upon the completion of training, the established VTP was deployed to plan for an independent set of 50 testing patient cases. Connecting the established VTP with the Eclipse workstation via the application programming interface, we tested the performance the VTP in operating Eclipse TPS for automatic treatment planning with another two independent patient cases. Like a human planner, VTP kept adjusting the planning objectives/constraints to improve plan quality until the plan was acceptable or the maximum number of adjustment steps was reached under both scenarios. The generated plans were evaluated using the ProKnow scoring system. The mean plan score (± standard deviation) of the 50 testing cases were improved from 6.18 ± 1.75 to 8.14 ± 1.27 by the VTP, with 9 being the maximal score. As for the two cases under Eclipse dose optimization, the plan scores were improved from 8 to 8.4 and 8.7 respectively by the VTP. These results indicated that the proposed DRL-based VTP was able to operate the in-house dose-volume constrained TPS and Eclipse TPS to automatically generate high-quality treatment plans for prostate cancer IMRT.

Towards a safe and efficient clinical implementation of machine learning in radiation oncology by exploring model interpretability, explainability and data-model dependency

The interest in machine learning (ML) has grown tremendously in recent years, partly due to the performance leap that occurred with new techniques of deep learning, convolutional neural networks for images, increased computational power, and wider availability of large datasets. Most fields of medicine follow that popular trend and, notably, radiation oncology is one of those that are at the forefront, with already a long tradition in using digital images and fully computerized workflows. ML models are driven by data, and in contrast with many statistical or physical models, they can be very large and complex, with countless generic parameters. This inevitably raises two questions, namely, the tight dependence between the models and the datasets that feed them, and the interpretability of the models, which scales with its complexity. Any problems in the data used to train the model will be later reflected in their performance. This, together with the low interpretability of ML models, makes their implementation into the clinical workflow particularly difficult. Building tools for risk assessment and quality assurance of ML models must involve then two main points: interpretability and data-model dependency. After a joint introduction of both radiation oncology and ML, this paper reviews the main risks and current solutions when applying the latter to workflows in the former. Risks associated with data and models, as well as their interaction, are detailed. Next, the core concepts of interpretability, explainability, and data-model dependency are formally defined and illustrated with examples. Afterwards, a broad discussion goes through key applications of ML in workflows of radiation oncology as well as vendors' perspectives for the clinical implementation of ML.

Modeling physician's preference in treatment plan approval of stereotactic body radiation therapy of prostate cancer

Objective.Treatment planning of radiation therapy is a time-consuming task. It is desirable to develop automatic planning approaches to generate plans favorable to physicians. The purpose of this study is to develop a deep learning based virtual physician network (VPN) that models physician's preference on plan approval for prostate cancer stereotactic body radiation therapy (SBRT).Approach.VPN takes one planning target volume (PTV) and eight organs at risk structure images, as well as a dose distribution of a plan seeking approval as input. It outputs a probability of approving the plan, and a dose distribution indicating improvements to the input dose. Due to the lack of unapproved plans in our database, VPN is trained using an adversarial framework. 68 prostate cancer patients who received 45Gyin 5-fraction SBRT were selected in this study, with 60 patients for training and cross validation, and 8 patients for independent testing.Main results.The trained VPN was able to differentiate approved and unapproved plans with Area under the curve 0.97 for testing data. For unapproved plans, after applying VPN's suggested dose improvement, the improved dose agreed with ground truth with relative differences2.03±2.17%for PTVD98%,0.49±0.29%for PTVV95%,3.08±2.24%for penile bulbDmean,3.73±2.20%for rectumV50%,and2.06±1.73%for bladderV50%.Significance.VPN was developed to accurately model a physician's preference on plan approval and to provide suggestions on how to improve the dose distribution.

Toward automatic beam angle selection for pencil-beam scanning proton liver Treatments: A deep learning-based approach

BACKGROUND

Dose deposition characteristics of proton radiation can be advantageous over photons. Proton treatment planning however poses additional challenges for the planners. Proton therapy is usually delivered with only a small number of beam angles, and the quality of a proton treatment plan is largely determined by the beam angles employed. Finding the optimal beam angles for a proton treatment plan requires time and experience, motivating the investigation of automatic beam angle selection methods.

PURPOSE

A deep learning-based approach to automatic beam angle selection is proposed for proton pencil-beam scanning treatment planning of liver lesions.

METHODS

We cast beam-angle selection as a multi-label classification problem. To account for angular boundary discontinuity, the underlying convolution neural network is trained with the proposed Circular Earth Mover's Distance based regularization and multi-label circular-smooth label technique. Furthermore, an analytical algorithm emulating proton treatment planners' clinical practice is employed in post-processing to improve the output of the model. Forty-nine patients that received proton liver treatments between 2017 and 2020 were randomly divided into training (n = 31), validation (n = 7), and test sets (n = 11). AI-selected beam angles were compared with those angles selected by human planners, and the dosimetric outcome was investigated by creating plans using knowledge-based treatment planning.

RESULTS

For 7 of the 11 cases in the test set, AI-selected beam angles agreed with those chosen by human planners to within 20 degrees (median angle difference = 10°; mean = 18.6°). Moreover, out of the total 22 beam angles predicted by the model, 15 (68%) were within 10 degrees of the human-selected angles. The high correlation in beam angles resulted in comparable dosimetric statistics between proton treatment plans generated using AI- and human-selected angles. For the cases with beam angle differences exceeding 20°, the dosimetric analysis showed similar plan quality although with different emphases on organ-at-risk sparing.

CONCLUSIONS

This pilot study demonstrated the feasibility of a novel deep learning-based beam angle selection technique. Testing on liver cancer patients showed that the resulting plans were clinically viable with comparable dosimetric quality to those using human-selected beam angles. In tandem with auto-contouring and knowledge-based treatment planning tools, the proposed model could represent a pathway for nearly fully automated treatment planning in proton therapy. This article is protected by copyright. All rights reserved.

Automatic Inverse Treatment Planning of Gamma Knife Radiosurgery via Deep Reinforcement Learning

PURPOSE

Several inverse planning algorithms have been developed for Gamma Knife (GK) radiosurgery to determine a large number of plan parameters via solving an optimization problem, which typically consists of multiple objectives. The priorities among these objectives need to be repetitively adjusted to achieve a clinically good plan for each patient. This study aimed to achieve automatic and intelligent priority-tuning, by developing a deep reinforcement learning (DRL) based method to model the tuning behaviors of human planners.

METHODS

We built a priority-tuning policy network using deep convolutional neural networks. Its input was a vector composed of multiple plan metrics that were used in our institution for GK plan evaluation. The network can determine which tuning action to take, based on the observed quality of the intermediate plan. We trained the network using an end-to-end DRL framework to approximate the optimal action-value function. A scoring function was designed to measure the plan quality to calculate the received reward of a tuning action.

RESULTS

Vestibular schwannoma was chosen as the test bed in this study. The number of training, validation and testing cases were 5, 5, and 16, respectively. For these three datasets, the average scores of the initial plans obtained with a same initial priority set were 3.63 ± 1.34, 3.83 ± 0.86 and 4.20 ± 0.78, respectively, while can be improved to 5.28 ± 0.23, 4.97 ± 0.44 and 5.22 ± 0.26 through manual priority tuning by human expert planners. Our network achieved competitive results with 5.42 ± 0.11, 5.10 ± 0. 42, 5.28 ± 0.20, respectively.

CONCLUSIONS

Our network can generate GK plans of comparable or slightly higher quality comparing with the plans generated by human planners via manual priority tuning for vestibular schwannoma cases. The network can potentially be incorporated into the clinical workflow as a planning assistance to improve GK planning efficiency and help to reduce plan quality variation caused by inter-planner variability. We also hope that our method can reduce the workload of GK planners and allow them to spend more time on more challenging cases. This article is protected by copyright. All rights reserved.

Assessing the robustness of artificial intelligence powered planning tools in radiotherapy clinical settings-a phantom simulation approach

Background

Artificial intelligence (AI) based radiotherapy treatment planning tools have gained interest in automating the treatment planning process. It is essential to understand their overall robustness in various clinical scenarios. This is an existing gap between many AI based tools and their actual clinical deployment. This study works to fill the gap for AI based treatment planning by investigating a clinical robustness assessment (CRA) tool for the AI based planning methods using a phantom simulation approach.

Methods

A cylindrical phantom was created in the treatment planning system (TPS) with the axial dimension of 30 cm by 18 cm. Key structures involved in pancreas stereotactic body radiation therapy (SBRT) including PTV25, PTV33, C-Loop, stomach, bowel and liver were created within the phantom. Several simulation scenarios were created to mimic multiple scenarios of anatomical changes, including displacement, expansion, rotation and combination of three. The goal of treatment planning was to deliver 25 Gy to PTV25 and 33 Gy to PTV33 in 5 fractions in simultaneous integral boost (SIB) manner while limiting luminal organ-at-risk (OAR) max dose to be under 29 Gy. A previously developed deep learning based AI treatment planning tool for pancreas SBRT was identified as the validation object. For each scenario, the anatomy information was fed into the AI tool and the final fluence map associated to the plan was generated, which was subsequently sent to TPS for leaf sequencing and dose calculation. The final auto plan's quality was analyzed against the treatment planning constraint. The final plans' quality was further analyzed to evaluate potential correlation with anatomical changes using the Manhattan plot.

Results

A total of 32 scenarios were simulated in this study. For all scenarios, the mean PTV25 V25Gy of the AI based auto plans was 96.7% while mean PTV33 V33Gy was 82.2%. Large variation (16.3%) in PTV33 V33Gy was observed due to anatomical variations, a.k.a. proximity of luminal structure to PTV33. Mean max dose was 28.55, 27.68 and 24.63 Gy for C-Loop, bowel and stomach, respectively. Using D_0.03cc as max dose surrogate, the value was 28.03, 27.12 and 23.84 Gy for C-Loop, bowel and stomach, respectively. Max dose constraint of 29 Gy was achieved for 81.3% cases for C-Loop and stomach, and 78.1% for bowel. Using D_0.03cc as max dose surrogate, the passing rate was 90.6% for C-Loop, and 81.3% for bowel and stomach. Manhattan plot revealed high correlation between the OAR over dose and the minimal distance between the PTV33 and OAR.

Conclusions

The results showed promising robustness of the pancreas SBRT AI tool, providing important evidence of its readiness for clinical implementation. The established workflow could guide the process of assuring clinical readiness of future AI based treatment planning tools.

Artificial intelligence applications in intensity modulated radiation treatment planning: an overview

Artificial intelligence (AI) refers to methods that improve and automate challenging human tasks by systematically capturing and applying relevant knowledge in these tasks. Over the past decades, a number of approaches have been developed to address different types and needs of system intelligence ranging from search strategies to knowledge representation and inference to robotic planning. In the context of radiation treatment planning, multiple AI approaches may be adopted to improve the planning quality and efficiency. For example, knowledge representation and inference methods may improve dose prescription by integrating and reasoning about the domain knowledge described in many clinical guidelines and clinical trials reports. In this review, we will focus on the most studied AI approach in intensity modulated radiation therapy (IMRT)/volumetric modulated arc therapy (VMAT)-machine learning (ML) and describe our recent efforts in applying ML to improve the quality, consistency, and efficiency of IMRT/VMAT planning. With the available high-quality data, we can build models to accurately predict critical variables for each step of the planning process and thus automate and improve its outcomes. Specific to the IMRT/VMAT planning process, we can build models for each of the four critical components in the process: dose-volume histogram (DVH), Dose, Fluence, and Human Planner. These models can be divided into two general groups. The first group focuses on encoding prior experience and knowledge through ML and more recently deep learning (DL) from prior clinical plans and using these models to predict the optimal DVH (DVH prediction model), or 3D dose distribution (dose prediction model), or fluence map (fluence map model). The goal of these models is to reduce or remove the trial-and-error process and guarantee consistently high-quality plans. The second group of models focuses on mimicking human planners' decision-making process (planning strategy model) during the iterative adjustments/guidance of the optimization engine. Each critical step of the IMRT/VMAT treatment planning process can be improved and automated by AI methods. As more training data becomes available and more sophisticated models are developed, we can expect that the AI methods in treatment planning will continue to improve accuracy, efficiency, and robustness.

Reinforcement Learning for Precision Oncology

Precision oncology is grounded in the increasing understanding of genetic and molecular mechanisms that underly malignant disease and offer different treatment pathways for the individual patient. The growing complexity of medical data has led to the implementation of machine learning techniques that are vastly applied for risk assessment and outcome prediction using either supervised or unsupervised learning. Still largely overlooked is reinforcement learning (RL) that addresses sequential tasks by exploring the underlying dynamics of an environment and shaping it by taking actions in order to maximize cumulative rewards over time, thereby achieving optimal long-term outcomes. Recent breakthroughs in RL demonstrated remarkable results in gameplay and autonomous driving, often achieving human-like or even superhuman performance. While this type of machine learning holds the potential to become a helpful decision support tool, it comes with a set of distinctive challenges that need to be addressed to ensure applicability, validity and safety. In this review, we highlight recent advances of RL focusing on studies in oncology and point out current challenges and pitfalls that need to be accounted for in future studies in order to successfully develop RL-based decision support systems for precision oncology.

A hierarchical deep reinforcement learning framework for intelligent automatic treatment planning of prostate cancer intensity modulated radiation therapy

Purpose.We have previously proposed an intelligent automatic treatment planning (IATP) framework that builds a virtual treatment planner network (VTPN) to operate a treatment planning system (TPS) to generate high-quality radiation therapy (RT) treatment plans. While the potential of IATP in automating RT treatment planning has been demonstrated, its poor scalability caused by an almost linear growth of network size with the number of treatment planning parameters (TPPs) is a bottleneck, preventing its application in complicate, but clinically relevant treatment planning problems. The decision-making behavior of the trained network is hard to understand. Motivated by the decision-making process of a human planner, this study proposes a hierarchical IATP framework.Methods and materials.The hierarchical VTPN (HieVTPN) consists of three networks, i.e. Structure-Net, Parameter-Net, and Action-Net. When interacting with a TPS, the networks are employed in a sequential order in each step to decide the structure to adjust, the TPP to adjust for the selected structure, and the specific adjustment manner for the parameter, respectively. We developed an end-to-end hierarchical deep reinforcement learning scheme to simultaneously train the three networks. We then evaluated the effectiveness of the proposed framework in the treatment planning problems for prostate cancer intensity modulated RT (IMRT) and stereotactic body RT (SBRT). We benchmarked the performance of our approach by comparing plans made by VTPN of a parallel architecture, and the human plans submitted for competition in the 2016 American Association of Medical Dosimetrist (AAMD)/Radiosurgery Society (RSS) Plan Study. We analyzed scalability of the network size with respect to the number of TPPs. Numerical experiments were also performed to understand the rationale of the decision-making behaviors of the trained HieVTPN.Results.Both HieVTPNs for prostate IMRT and SBRT were trained successfully using 10 training patient cases and 5 validation cases. For IMRT, HieVTPN was able to generate high-quality plans for 59 testing patient cases that were not included in training process, achieving an average plan score of 8.62 (±0.83), with 9 being the maximal score. The score was comparable to that of the VTPN, 8.45 (±0.48). For SBRT planning, HieVTPN achieved an average plan score of 139.07 on five testing patient cases compared to the score of 132.21 averaged over the human plans summited for competition in AAMD/RSS plan study. Different from VTPN with network size linearly scaling with the number of TPPs, the network size of HieVTPN is almost independent of the number of TPPs. It was also observed that the decision-making behaviors of HieVTPN were understandable and generally agreed with the human experience.Conclusions.With the scalability and explainability, the hierarchical IATP framework is more favorable than the previous framework in terms of handling treatment planning problems involving a large number of TPPs.

Artificial intelligence and machine learning for medical imaging: A technology review

Artificial intelligence (AI) has recently become a very popular buzzword, as a consequence of disruptive technical advances and impressive experimental results, notably in the field of image analysis and processing. In medicine, specialties where images are central, like radiology, pathology or oncology, have seized the opportunity and considerable efforts in research and development have been deployed to transfer the potential of AI to clinical applications. With AI becoming a more mainstream tool for typical medical imaging analysis tasks, such as diagnosis, segmentation, or classification, the key for a safe and efficient use of clinical AI applications relies, in part, on informed practitioners. The aim of this review is to present the basic technological pillars of AI, together with the state-of-the-art machine learning methods and their application to medical imaging. In addition, we discuss the new trends and future research directions. This will help the reader to understand how AI methods are now becoming an ubiquitous tool in any medical image analysis workflow and pave the way for the clinical implementation of AI-based solutions.