Automated prediction of dosimetric eligibility of patients with prostate cancer undergoing intensity-modulated radiation therapy using a convolutional neural network

Deep-neural network approaches for predicting 3D dose distribution in intensity-modulated radiotherapy of the brain tumors

PURPOSE

The aim of this study is to reduce treatment planning time by predicting the intensity-modulated radiotherapy 3D dose distribution using deep learning for brain cancer patients. "For this purpose, two different approaches in dose prediction, i.e., first only planning target volume (PTV) and second PTV with organs at risk (OARs) as input of the U-net model, are employed and their results are compared."

METHODS AND MATERIALS

The data of 99 patients with glioma tumors referred for IMRT treatment were used so that the images of 90 patients were regarded as training datasets and the others were for the test. All patients were manually planned and treated with sixth-field IMRT; the photon energy was 6MV. The treatment plans were done with the Collapsed Cone Convolution algorithm to deliver 60 Gy in 30 fractions.

RESULTS

The obtained accuracy and similarity for the proposed methods in dose prediction when compared to the clinical dose distributions on test patients according to MSE, dice metric and SSIM for the Only-PTV and PTV-OARs methods are on average (0.05, 0.851, 0.83) and (0.056, 0.842, 0.82) respectively. Also, dose prediction is done in an extremely short time.

CONCLUSION

The same results of the two proposed methods prove that the presence of OARs in addition to PTV does not provide new knowledge to the network and only by defining the PTV and its location in the imaging slices, does the dose distribution become predictable. Therefore, the Only-PTV method by eliminating the process of introducing OARs can reduce the overall designing time of treatment by IMRT in patients with glioma tumors.

Prediction of plan adaptation in head and neck cancer proton therapy using clinical, radiographic, and dosimetric features

PURPOSE

Because proton head and neck (HN) treatments are sensitive to anatomical changes, plan adaptation (re-plan) during the treatment course is needed for a significant portion of patients. We aim to predict re-plan at plan review stage for HN proton therapy with a neural network (NN) model trained with patients' dosimetric and clinical features. The model can serve as a valuable tool for planners to assess the probability of needing to revise the current plan.

METHODS AND MATERIALS

Mean beam dose heterogeneity index (BHI), defined as the ratio of the maximum beam dose to the prescription dose, plan robustness features (clinical target volume (CTV), V100 changes, and V100 > 95% passing rates in 21 robust evaluation scenarios), as well as clinical features (e.g., age, tumor site, and surgery/chemotherapy status) were gathered from 171 patients treated at our proton center in 2020, with a median age of 64 and stages from I-IVc across 13 HN sites. Statistical analyses of dosimetric parameters and clinical features were conducted between re-plan and no-replan groups. A NN was trained and tested using these features. Receiver operating characteristic (ROC) analysis was conducted to evaluate the performance of the prediction model. A sensitivity analysis was done to determine feature importance.

RESULTS

Mean BHI in the re-plan group was significantly higher than the no-replan group (p < .01). Tumor site (p < .01), chemotherapy status (p < .01), and surgery status (p < .01) were significantly correlated to re-plan. The model had sensitivities/specificities of 75.0%/77.4%, respectively, and an area under the ROC curve of .855.

CONCLUSION

There are several dosimetric and clinical features that correlate to re-plans, and NNs trained with these features can be used to predict HN re-plans, which can be used to reduce re-plan rate by improving plan quality.

A Survey on Deep Learning for Precision Oncology

Precision oncology, which ensures optimized cancer treatment tailored to the unique biology of a patient’s disease, has rapidly developed and is of great clinical importance. Deep learning has become the main method for precision oncology. This paper summarizes the recent deep-learning approaches relevant to precision oncology and reviews over 150 articles within the last six years. First, we survey the deep-learning approaches categorized by various precision oncology tasks, including the estimation of dose distribution for treatment planning, survival analysis and risk estimation after treatment, prediction of treatment response, and patient selection for treatment planning. Secondly, we provide an overview of the studies per anatomical area, including the brain, bladder, breast, bone, cervix, esophagus, gastric, head and neck, kidneys, liver, lung, pancreas, pelvis, prostate, and rectum. Finally, we highlight the challenges and discuss potential solutions for future research directions.

A Synopsis of Machine and Deep Learning in Medical Physics and Radiology

Machine learning (ML) and deep learning (DL) technologies introduced in the fields of medical physics, radiology, and oncology have made great strides in the past few years. A good many applications have proven to be an efficacious automated diagnosis and radiotherapy system. This paper outlines DL's general concepts and principles, key computational methods, and resources, as well as the implementation of automated models in diagnostic radiology and radiation oncology research. In addition, the potential challenges and solutions of DL technology are also discussed.

Transfer learning for medical image classification: a literature review

BACKGROUND

Transfer learning (TL) with convolutional neural networks aims to improve performances on a new task by leveraging the knowledge of similar tasks learned in advance. It has made a major contribution to medical image analysis as it overcomes the data scarcity problem as well as it saves time and hardware resources. However, transfer learning has been arbitrarily configured in the majority of studies. This review paper attempts to provide guidance for selecting a model and TL approaches for the medical image classification task.

METHODS

425 peer-reviewed articles were retrieved from two databases, PubMed and Web of Science, published in English, up until December 31, 2020. Articles were assessed by two independent reviewers, with the aid of a third reviewer in the case of discrepancies. We followed the PRISMA guidelines for the paper selection and 121 studies were regarded as eligible for the scope of this review. We investigated articles focused on selecting backbone models and TL approaches including feature extractor, feature extractor hybrid, fine-tuning and fine-tuning from scratch.

RESULTS

The majority of studies (n = 57) empirically evaluated multiple models followed by deep models (n = 33) and shallow (n = 24) models. Inception, one of the deep models, was the most employed in literature (n = 26). With respect to the TL, the majority of studies (n = 46) empirically benchmarked multiple approaches to identify the optimal configuration. The rest of the studies applied only a single approach for which feature extractor (n = 38) and fine-tuning from scratch (n = 27) were the two most favored approaches. Only a few studies applied feature extractor hybrid (n = 7) and fine-tuning (n = 3) with pretrained models.

CONCLUSION

The investigated studies demonstrated the efficacy of transfer learning despite the data scarcity. We encourage data scientists and practitioners to use deep models (e.g. ResNet or Inception) as feature extractors, which can save computational costs and time without degrading the predictive power.

Accuracy Improvement Method Based on Characteristic Database Classification for IMRT Dose Prediction in Cervical Cancer: Scientifically Training Data Selection

Purpose

Consistent training and testing datasets can lead to good performance for deep learning (DL) models. However, a large high-quality training dataset for unusual clinical scenarios is usually not easy to collect. The work aims to find optimal training data collection strategies for DL-based dose prediction models.

Materials and Methods

A total of 325 clinically approved cervical IMRT plans were utilized. We designed comparison experiments to investigate the impact of (1) beam angles, (2) the number of beams, and (3) patient position for DL dose prediction models. In addition, a novel geometry-based beam mask generation method was proposed to provide beam setting information in the model training process. What is more, we proposed a new training strategy named “full-database pre-trained strategy”.

Results

The model trained with a homogeneous dataset with the same beam settings achieved the best performance [mean prediction errors of planning target volume (PTV), bladder, and rectum: 0.29 ± 0.15%, 3.1 ± 2.55%, and 3.15 ± 1.69%] compared with that trained with large mixed beam setting plans (mean errors of PTV, bladder, and rectum: 0.8 ± 0.14%, 5.03 ± 2.2%, and 4.45 ± 1.4%). A homogeneous dataset is more accessible to train an accurate dose prediction model (mean errors of PTV, bladder and rectum: 2.2 ± 0.15%, 5 ± 2.1%, and 3.23 ± 1.53%) than a non-homogeneous one (mean errors of PTV, bladder and rectum: 2.55 ± 0.12%, 6.33 ± 2.46%, and 4.76 ± 2.91%) without other processing approaches. The added beam mask can constantly improve the model performance, especially for datasets with different beam settings (mean errors of PTV, bladder, and rectum improved from 0.8 ± 0.14%, 5.03 ± 2.2%, and 4.45 ± 1.4% to 0.29 ± 0.15%, 3.1 ± 2.55%, and 3.15 ± 1.69%).

Conclusions

A consistent dataset is recommended to form a patient-specific IMRT dose prediction model. When a consistent dataset is not accessible to collect, a large dataset with different beam angles and a training model with beam information can also get a relatively good model. The full-database pre-trained strategies can rapidly form an accuracy model from a pre-trained model. The proposed beam mask can effectively improve the model performance. Our study may be helpful for further dose prediction studies in terms of training strategies or database establishment.

Dose Prediction Using a Three-Dimensional Convolutional Neural Network for Nasopharyngeal Carcinoma With Tomotherapy

Purpose

This study focused on predicting 3D dose distribution at high precision and generated the prediction methods for nasopharyngeal carcinoma patients (NPC) treated with Tomotherapy based on the patient-specific gap between organs at risk (OARs) and planning target volumes (PTVs).

Methods

A convolutional neural network (CNN) is trained using the CT and contour masks as the input and dose distributions as output. The CNN is based on the "3D Dense-U-Net", which combines the U-Net and the Dense-Net. To evaluate the model, we retrospectively used 124 NPC patients treated with Tomotherapy, in which 96 and 28 patients were randomly split and used for model training and test, respectively. We performed comparison studies using different training matrix shapes and dimensions for the CNN models, i.e., 128 ×128 ×48 (for Model I), 128 ×128 ×16 (for Model II), and 2D Dense U-Net (for Model III). The performance of these models was quantitatively evaluated using clinically relevant metrics and statistical analysis.

Results

We found a more considerable height of the training patch size yields a better model outcome. The study calculated the corresponding errors by comparing the predicted dose with the ground truth. The mean deviations from the mean and maximum doses of PTVs and OARs were 2.42 and 2.93%. Error for the maximum dose of right optic nerves in Model I was 4.87 ± 6.88%, compared with 7.9 ± 6.8% in Model II (p=0.08) and 13.85 ± 10.97% in Model III (p<0.01); the Model I performed the best. The gamma passing rates of PTV₆₀ for 3%/3 mm criteria was 83.6 ± 5.2% in Model I, compared with 75.9 ± 5.5% in Model II (p<0.001) and 77.2 ± 7.3% in Model III (p<0.01); the Model I also gave the best outcome. The prediction error of D₉₅ for PTV₆₀ was 0.64 ± 0.68% in Model I, compared with 2.04 ± 1.38% in Model II (p<0.01) and 1.05 ± 0.96% in Model III (p=0.01); the Model I was also the best one.

Conclusions

It is significant to train the dose prediction model by exploiting deep-learning techniques with various clinical logic concepts. Increasing the height (Y direction) of training patch size can improve the dose prediction accuracy of tiny OARs and the whole body. Our dose prediction network model provides a clinically acceptable result and a training strategy for a dose prediction model. It should be helpful to build automatic Tomotherapy planning.

Clinical Implementation of Automated Treatment Planning for Rectum Intensity-Modulated Radiotherapy Using Voxel-Based Dose Prediction and Post-Optimization Strategies

Purpose

This study aims to demonstrate the feasibility of clinical implementation of automated treatment planning (ATP) using voxel-based dose prediction and post-optimization strategies for rectal cancer on uRT (United Imaging Healthcare, Shanghai, China) treatment planning system.

Methods

A total of 180 previously treated rectal cancer cases were enrolled in this study, including 160 cases for training, 10 for validation and 10 for testing. Using CT image data, planning target volumes (PTVs) and contour delineation of the organs at risk (OARs) as input and three-dimensional (3D) dose distribution as output, a 3D-Uet DL model was developed. Based on the voxel-wise prediction dose distribution, intensity-modulated radiation therapy (IMRT) plans were then generated automatedly using post-optimization strategies, including a complex clinical dose target metrics homogeneity index (HI) and conformation index (CI). To evaluate the performance of the proposed ATP approach, the dose-volume histogram (DVH) parameters of OARs and PTV and the 3D dose distributions of the plan were compared with those of manual plans.

Results

By combining clinical post-optimization strategies, the automatically generated treatment plan can achieve better homogeneous PTV coverage and dose sparing for OARs except the mean dose for femoral-head compared with the use of the mean square error objective function alone. Compared with the manual plan, no statistically significant differences in HI, CI or global maximum dose were found. The manual plans perform slightly better than plans with post-optimization strategies in other dosimetric indexes, but these plans are still within clinical requirements.

Conclusions

With the help of clinical post-optimization strategies, the proposed new ATP solution can generate IMRT plans that are within clinically acceptable levels and comparable to plans manually generated by dosimetrists.

Convolutional neural network and transfer learning for dose volume histogram prediction for prostate cancer radiotherapy

To adopt a transfer learning approach and establish a convolutional neural network (CNN) model for the prediction of rectum and bladder dose-volume histograms (DVH) in prostate patients treated with a VMAT technique. One hundred forty-four VMAT patients with intermediate or high-risk prostate cancer were included in this study. Data were split into two sets: 120 and 24 patients, respectively. The second set was used for final validation. To ensure the accuracy of the training data, we developed a ground-truth analysis for detecting and correcting for all potential outliers. We used transfer learning in combination with a pre-trained VGG-16 network. We dropped the fully connected layers from the VGG-16 and added a new fully connected neural network. The inputs for the CNN were a 2D image of the volumes contoured in the CT, but we only retained the geometrical information of every CT-slice. The outputs were the corresponding rectum and bladder DVH for every slice. We used a confusion matrix to analyze the performance of our model. Our model achieved 100% and 81% of true positive and true negative predictions, respectively. We have an overall accuracy of 87.5%, a misclassification rate of 12.5%, and a precision of 100%. We have successfully developed a model for reliable prediction of rectum and bladder DVH in prostate patients by applying a previously pre-trained CNN. To our knowledge, this is the first attempt to apply transfer learning to the prediction of DVHs that accounts for the ground truth problem.

A scoping review of transfer learning research on medical image analysis using ImageNet

OBJECTIVE

Employing transfer learning (TL) with convolutional neural networks (CNNs), well-trained on non-medical ImageNet dataset, has shown promising results for medical image analysis in recent years. We aimed to conduct a scoping review to identify these studies and summarize their characteristics in terms of the problem description, input, methodology, and outcome.

MATERIALS AND METHODS

To identify relevant studies, MEDLINE, IEEE, and ACM digital library were searched for studies published between June 1st^, 2012 and January 2nd, 2020. Two investigators independently reviewed articles to determine eligibility and to extract data according to a study protocol defined a priori.

RESULTS

After screening of 8421 articles, 102 met the inclusion criteria. Of 22 anatomical areas, eye (18%), breast (14%), and brain (12%) were the most commonly studied. Data augmentation was performed in 72% of fine-tuning TL studies versus 15% of the feature-extracting TL studies. Inception models were the most commonly used in breast related studies (50%), while VGGNet was the common in eye (44%), skin (50%) and tooth (57%) studies. AlexNet for brain (42%) and DenseNet for lung studies (38%) were the most frequently used models. Inception models were the most frequently used for studies that analyzed ultrasound (55%), endoscopy (57%), and skeletal system X-rays (57%). VGGNet was the most common for fundus (42%) and optical coherence tomography images (50%). AlexNet was the most frequent model for brain MRIs (36%) and breast X-Rays (50%). 35% of the studies compared their model with other well-trained CNN models and 33% of them provided visualization for interpretation.

DISCUSSION

This study identified the most prevalent tracks of implementation in the literature for data preparation, methodology selection and output evaluation for various medical image analysis tasks. Also, we identified several critical research gaps existing in the TL studies on medical image analysis. The findings of this scoping review can be used in future TL studies to guide the selection of appropriate research approaches, as well as identify research gaps and opportunities for innovation.

A Review on Application of Deep Learning Algorithms in External Beam Radiotherapy Automated Treatment Planning

Treatment planning plays an important role in the process of radiotherapy (RT). The quality of the treatment plan directly and significantly affects patient treatment outcomes. In the past decades, technological advances in computer and software have promoted the development of RT treatment planning systems with sophisticated dose calculation and optimization algorithms. Treatment planners now have greater flexibility in designing highly complex RT treatment plans in order to mitigate the damage to healthy tissues better while maximizing radiation dose to tumor targets. Nevertheless, treatment planning is still largely a time-inefficient and labor-intensive process in current clinical practice. Artificial intelligence, including machine learning (ML) and deep learning (DL), has been recently used to automate RT treatment planning and has gained enormous attention in the RT community due to its great promises in improving treatment planning quality and efficiency. In this article, we reviewed the historical advancement, strengths, and weaknesses of various DL-based automated RT treatment planning techniques. We have also discussed the challenges, issues, and potential research directions of DL-based automated RT treatment planning techniques.

Multicriterial CNN based beam generation for robotic radiosurgery of the prostate

Although robotic radiosurgery offers a flexible arrangement of treatment beams, generating treatment plans is computationally challenging and a time consuming process for the planner. Furthermore, different clinical goals have to be considered during planning and generally different sets of beams correspond to different clinical goals. Typically, candidate beams sampled from a randomized heuristic form the basis for treatment planning. We propose a new approach to generate candidate beams based on deep learning using radiological features as well as the desired constraints. We demonstrate that candidate beams generated for specific clinical goals can improve treatment plan quality. Furthermore, we compare two approaches to include information about constraints in the prediction. Our results show that CNN generated beams can improve treatment plan quality for different clinical goals, increasing coverage from 91.2 to 96.8% for 3,000 candidate beams on average. When including the clinical goal in the training, coverage is improved by 1.1% points.

The Application of Artificial Intelligence in Prostate Cancer Management—What Improvements Can Be Expected? A Systematic Review

Artificial Intelligence (AI) is progressively remodeling our daily life. A large amount of information from “big data” now enables machines to perform predictions and improve our healthcare system. AI has the potential to reshape prostate cancer (PCa) management thanks to growing applications in the field. The purpose of this review is to provide a global overview of AI in PCa for urologists, pathologists, radiotherapists, and oncologists to consider future changes in their daily practice. A systematic review was performed, based on PubMed MEDLINE, Google Scholar, and DBLP databases for original studies published in English from January 2009 to January 2019 relevant to PCa, AI, Machine Learning, Artificial Neural Networks, Convolutional Neural Networks, and Natural-Language Processing. Only articles with full text accessible were considered. A total of 1008 articles were reviewed, and 48 articles were included. AI has potential applications in all fields of PCa management: analysis of genetic predispositions, diagnosis in imaging, and pathology to detect PCa or to differentiate between significant and non-significant PCa. AI also applies to PCa treatment, whether surgical intervention or radiotherapy, skills training, or assessment, to improve treatment modalities and outcome prediction. AI in PCa management has the potential to provide a useful role by predicting PCa more accurately, using a multiomic approach and risk-stratifying patients to provide personalized medicine.

Artificial intelligence in radiotherapy: a technological review

Radiation therapy (RT) is widely used to treat cancer. Technological advances in RT have occurred in the past 30 years. These advances, such as three-dimensional image guidance, intensity modulation, and robotics, created challenges and opportunities for the next breakthrough, in which artificial intelligence (AI) will possibly play important roles. AI will replace certain repetitive and labor-intensive tasks and improve the accuracy and consistency of others, particularly those with increased complexity because of technological advances. The improvement in efficiency and consistency is important to manage the increasing cancer patient burden to the society. Furthermore, AI may provide new functionalities that facilitate satisfactory RT. The functionalities include superior images for real-time intervention and adaptive and personalized RT. AI may effectively synthesize and analyze big data for such purposes. This review describes the RT workflow and identifies areas, including imaging, treatment planning, quality assurance, and outcome prediction, that benefit from AI. This review primarily focuses on deep-learning techniques, although conventional machine-learning techniques are also mentioned.

Artificial intelligence in radiotherapy

Artificial intelligence (AI) has already been implemented widely in the medical field in the recent years. This paper first reviews the background of AI and radiotherapy. Then it explores the basic concepts of different AI algorithms and machine learning methods, such as neural networks, that are available to us today and how they are being implemented in radiotherapy and diagnostic processes, such as medical imaging, treatment planning, patient simulation, quality assurance and radiation dose delivery. It also explores the ongoing research on AI methods that are to be implemented in radiotherapy in the future. The review shows very promising progress and future for AI to be widely used in various areas of radiotherapy. However, basing on various concerns such as availability and security of using big data, and further work on polishing and testing AI algorithms, it is found that we may not ready to use AI primarily in radiotherapy at the moment.

Feasibility and analysis of CNN-based candidate beam generation for robotic radiosurgery

PURPOSE

Robotic radiosurgery offers the flexibility of a robotic arm to enable high conformity to the target and a steep dose gradient. However, treatment planning becomes a computationally challenging task as the search space for potential beam directions for dose delivery is arbitrarily large. We propose an approach based on deep learning to improve the search for treatment beams.

METHODS

In clinical practice, a set of candidate beams generated by a randomized heuristic forms the basis for treatment planning. We use a convolutional neural network to identify promising candidate beams. Using radiological features of the patient, we predict the influence of a candidate beam on the delivered dose individually and let this prediction guide the selection of candidate beams. Features are represented as projections of the organ structures which are relevant during planning. Solutions to the inverse planning problem are generated for random and CNN-predicted candidate beams.

RESULTS

The coverage increases from 95.35% to 97.67% for 6000 heuristically and CNN-generated candidate beams, respectively. Conversely, a similar coverage can be achieved for treatment plans with half the number of candidate beams. This results in a patient-dependent reduced averaged computation time of 20.28%-45.69%. The number of active treatment beams can be reduced by 11.35% on average, which reduces treatment time. Constraining the maximum number of candidate beams per beam node can further improve the average coverage by 0.75 percentage points for 6000 candidate beams.

CONCLUSIONS

We show that deep learning based on radiological features can substantially improve treatment plan quality, reduce computation runtime, and treatment time compared to the heuristic approach used in clinics.

Deep learning in medical imaging and radiation therapy

The goals of this review paper on deep learning (DL) in medical imaging and radiation therapy are to (a) summarize what has been achieved to date; (b) identify common and unique challenges, and strategies that researchers have taken to address these challenges; and (c) identify some of the promising avenues for the future both in terms of applications as well as technical innovations. We introduce the general principles of DL and convolutional neural networks, survey five major areas of application of DL in medical imaging and radiation therapy, identify common themes, discuss methods for dataset expansion, and conclude by summarizing lessons learned, remaining challenges, and future directions.