Explainable attention guided adversarial deep network for 3D radiotherapy dose distribution prediction

Deep learning-based dose prediction for magnetic resonance-guided prostate radiotherapy

BACKGROUND

Daily adaptive radiotherapy, as performed with the Elekta Unity MR-Linac, requires choosing between different adaptation methods, namely ATP (Adapt to Position) and ATS (Adapt to Shape), where the latter requires daily re-contouring to obtain a dose plan tailored to the daily anatomy. These steps are inherently resource-intensive, and quickly predicting the dose distribution and the dosimetric evaluation criteria while the patient is on the table could facilitate a fast selection of adaptation method and decrease the treatment times.

PURPOSE

In this work, we aimed to develop a deep-learning-based dose-prediction pipeline for prostate MR-Linac treatments.

METHODS

Two hundred twelve MR-images, structure sets, and dose distributions from 35 prostate patients treated with 6.1 Gy for 7 or 6 fractions at our MR-Linac were included, split into train/test partitions of 152/60 images, respectively. A deep-learning segmentation network was trained to segment the CTV (prostate), bladder, and rectum. A second network was trained to predict the dose distribution based on manually delineated structures. At inference, the predicted segmentations acted as input to the dose prediction network, and the predicted dose was compared to the true (optimized in the treatment planning system) dose distribution.

RESULTS

Median DSC values from the segmentation network were 0.90/0.94/0.87 for CTV/bladder/rectum. Predicted segmentations as input to the dose prediction resulted in mean differences between predicted and true doses of 0.7%/0.7%/1.7% (relative to the prescription dose) for D98%/D95%/D2% for the CTV. For the bladder, the difference was 0.7%/0.3% for Dmean/D2% and for the rectum 0.1/0.2/0.2 pp (percentage points) for V33Gy/V38Gy/V41Gy. In comparison, true segmentations as input resulted in differences of 1.1%/0.9%/1.6% for CTV, 0.5%/0.4% for bladder, and 0.7/0.4/0.3 pp for the rectum. Only D2% for CTV and Dmean/D2% for bladder were found to be statistically significantly better when using true structures instead of predicted structures as input to the dose prediction.

CONCLUSIONS

Small differences in the fulfillment of clinical dose-volume constraints are seen between utilizing deep-learning predicted structures as input to a dose prediction network and manual structures. Overall mean differences <2% indicate that the dose-prediction pipeline is useful as a decision support tool where differences are >2%.

FA-Net: A hierarchical feature fusion and interactive attention-based network for dose prediction in liver cancer patients

Dose prediction is a crucial step in automated radiotherapy planning for liver cancer. Several deep learning-based approaches for dose prediction have been proposed to enhance the design efficiency and quality of radiotherapy plan. However, these approaches usually take CT images and contours of organs at risk (OARs) and planning target volume (PTV) as a multi-channel input and is thus difficult to extract sufficient feature information from each input, which results in unsatisfactory dose distribution. In this paper, we propose a novel dose prediction network for liver cancer based on hierarchical feature fusion and interactive attention. A feature extraction module is first constructed to extract multi-scale features from different inputs, and a hierarchical feature fusion module is then built to fuse these multi-scale features hierarchically. A decoder based on attention mechanism is designed to gradually reconstruct the fused features into dose distribution. Additionally, we design an autoencoder network to generate a perceptual loss during training stage, which is used to improve the accuracy of dose prediction. The proposed method is tested on private clinical dataset and obtains HI and CI of 0.31 and 0.87, respectively. The experimental results are better than those by several existing methods, indicating that the dose distribution generated by the proposed method is close to that approved in clinics. The codes are available at https://github.com/hired-ld/FA-Net.

LENAS: Learning-Based Neural Architecture Search and Ensemble for 3-D Radiotherapy Dose Prediction

Radiation therapy treatment planning requires balancing the delivery of the target dose while sparing normal tissues, making it a complex process. To streamline the planning process and enhance its quality, there is a growing demand for knowledge-based planning (KBP). Ensemble learning has shown impressive power in various deep learning tasks, and it has great potential to improve the performance of KBP. However, the effectiveness of ensemble learning heavily depends on the diversity and individual accuracy of the base learners. Moreover, the complexity of model ensembles is a major concern, as it requires maintaining multiple models during inference, leading to increased computational cost and storage overhead. In this study, we propose a novel learning-based ensemble approach named LENAS, which integrates neural architecture search with knowledge distillation for 3-D radiotherapy dose prediction. Our approach starts by exhaustively searching each block from an enormous architecture space to identify multiple architectures that exhibit promising performance and significant diversity. To mitigate the complexity introduced by the model ensemble, we adopt the teacher-student paradigm, leveraging the diverse outputs from multiple learned networks as supervisory signals to guide the training of the student network. Furthermore, to preserve high-level semantic information, we design a hybrid loss to optimize the student network, enabling it to recover the knowledge embedded within the teacher networks. The proposed method has been evaluated on two public datasets: 1) OpenKBP and 2) AIMIS. Extensive experimental results demonstrate the effectiveness of our method and its superior performance to the state-of-the-art methods. Code: github.com/hust-linyi/LENAS.

Deep evidential learning for radiotherapy dose prediction

BACKGROUND

As we navigate towards integrating deep learning methods in the real clinic, a safety concern lies in whether and how the model can express its own uncertainty when making predictions. In this work, we present a novel application of an uncertainty-quantification framework called Deep Evidential Learning in the domain of radiotherapy dose prediction.

METHOD

Using medical images of the Open Knowledge-Based Planning Challenge dataset, we found that this model can be effectively harnessed to yield uncertainty estimates that inherited correlations with prediction errors upon completion of network training. This was achieved only after reformulating the original loss function for a stable implementation.

RESULTS

We found that (i) epistemic uncertainty was highly correlated with prediction errors, with various association indices comparable or stronger than those for Monte-Carlo Dropout and Deep Ensemble methods, (ii) the median error varied with uncertainty threshold much more linearly for epistemic uncertainty in Deep Evidential Learning relative to these other two conventional frameworks, indicative of a more uniformly calibrated sensitivity to model errors, (iii) relative to epistemic uncertainty, aleatoric uncertainty demonstrated a more significant shift in its distribution in response to Gaussian noise added to CT intensity, compatible with its interpretation as reflecting data noise.

CONCLUSION

Collectively, our results suggest that Deep Evidential Learning is a promising approach that can endow deep-learning models in radiotherapy dose prediction with statistical robustness. We have also demonstrated how this framework leads to uncertainty heatmaps that correlate strongly with model errors, and how it can be used to equip the predicted Dose-Volume-Histograms with confidence intervals.

Precision dose prediction for breast cancer patients undergoing IMRT: The Swin-UMamba-Channel Model

BACKGROUND

Radiation therapy is one of the crucial treatment modalities for cancer. An excellent radiation therapy plan relies heavily on an outstanding dose distribution map, which is traditionally generated through repeated trials and adjustments by experienced physicists. However, this process is both time-consuming and labor-intensive, and it comes with a degree of subjectivity. Now, with the powerful capabilities of deep learning, we are able to predict dose distribution maps more accurately, effectively overcoming these challenges.

METHODS

In this study, we propose a novel Swin-UMamba-Channel prediction model specifically designed for predicting the dose distribution of patients with left breast cancer undergoing radiotherapy after total mastectomy. This model integrates anatomical position information of organs and ray angle information, significantly enhancing prediction accuracy. Through iterative training of the generator (Swin-UMamba) and discriminator, the model can generate images that closely match the actual dose, assisting physicists in quickly creating DVH curves and shortening the treatment planning cycle. Our model exhibits excellent performance in terms of prediction accuracy, computational efficiency, and practicality, and its effectiveness has been further verified through comparative experiments with similar networks.

RESULTS

The results of the study indicate that our model can accurately predict the clinical dose of breast cancer patients undergoing intensity-modulated radiation therapy (IMRT). The predicted dose range is from 0 to 50 Gy, and compared with actual data, it shows a high accuracy with an average Dice similarity coefficient of 0.86. Specifically, the average dose change rate for the planning target volume ranges from 0.28 % to 1.515 %, while the average dose change rates for the right and left lungs are 2.113 % and 0.508 %, respectively. Notably, due to their small sizes, the heart and spinal cord exhibit relatively higher average dose change rates, reaching 3.208 % and 1.490 %, respectively. In comparison with similar dose studies, our model demonstrates superior performance. Additionally, our model possesses fewer parameters, lower computational complexity, and shorter processing time, further enhancing its practicality and efficiency. These findings provide strong evidence for the accuracy and reliability of our model in predicting doses, offering significant technical support for IMRT in breast cancer patients.

CONCLUSION

This study presents a novel Swin-UMamba-Channel dose prediction model, and its results demonstrate its precise prediction of clinical doses for the target area of left breast cancer patients undergoing total mastectomy and IMRT. These remarkable achievements provide valuable reference data for subsequent plan optimization and quality control, paving a new path for the application of deep learning in the field of radiation therapy.

Multi-sequence generative adversarial network: better generation for enhanced magnetic resonance imaging images

Introduction

MRI is one of the commonly used diagnostic methods in clinical practice, especially in brain diseases. There are many sequences in MRI, but T1CE images can only be obtained by using contrast agents. Many patients (such as cancer patients) must undergo alignment of multiple MRI sequences for diagnosis, especially the contrast-enhanced magnetic resonance sequence. However, some patients such as pregnant women, children, etc. find it difficult to use contrast agents to obtain enhanced sequences, and contrast agents have many adverse reactions, which can pose a significant risk. With the continuous development of deep learning, the emergence of generative adversarial networks makes it possible to extract features from one type of image to generate another type of image.

Methods

We propose a generative adversarial network model with multimodal inputs and end-to-end decoding based on the pix2pix model. For the pix2pix model, we used four evaluation metrics: NMSE, RMSE, SSIM, and PNSR to assess the effectiveness of our generated model.

Results

Through statistical analysis, we compared our proposed new model with pix2pix and found significant differences between the two. Our model outperformed pix2pix, with higher SSIM and PNSR, lower NMSE and RMSE. We also found that the input of T1W images and T2W images had better effects than other combinations, providing new ideas for subsequent work on generating magnetic resonance enhancement sequence images. By using our model, it is possible to generate magnetic resonance enhanced sequence images based on magnetic resonance non-enhanced sequence images.

Discussion

This has significant implications as it can greatly reduce the use of contrast agents to protect populations such as pregnant women and children who are contraindicated for contrast agents. Additionally, contrast agents are relatively expensive, and this generation method may bring about substantial economic benefits.

Three-dimensional dose prediction based on deep convolutional neural networks for brain cancer in CyberKnife: accurate beam modelling of homogeneous tissue

Objectives

Accurate beam modelling is essential for dose calculation in stereotactic radiation therapy (SRT), such as CyberKnife treatment. However, the present deep learning methods only involve patient anatomical images and delineated masks for training. These studies generally focus on traditional intensity-modulated radiation therapy (RT) plans. Nevertheless, this paper aims to develop a deep CNN-based method for CyberKnife plan dose prediction about brain cancer patients. It utilized modelled beam information, target delineation, and patient anatomical information.

Methods

This study proposes a method that adds beam information to predict the dose distribution of CyberKnife in brain cases. A retrospective dataset of 88 brain and abdominal cancer patients treated with the Ray-tracing algorithm was performed. The datasets include patients' anatomical information (planning CT), binary masks for organs at risk (OARs) and targets, and clinical plans (containing beam information). The datasets were randomly split into 68, 6, and 14 brain cases for training, validation, and testing, respectively.

Results

Our proposed method performs well in SRT dose prediction. First, for the gamma passing rates in brain cancer cases, with the 2 mm/2% criteria, we got 96.7% ± 2.9% for the body, 98.3% ± 3.0% for the planning target volume, and 100.0% ± 0.0% for the OARs with small volumes referring to the clinical plan dose. Secondly, the model predictions matched the clinical plan's dose-volume histograms reasonably well for those cases. The differences in key metrics at the target area were generally below 1.0 Gy (approximately a 3% difference relative to the prescription dose).

Conclusions

The preliminary results for selected 14 brain cancer cases suggest that accurate 3-dimensional dose prediction for brain cancer in CyberKnife can be accomplished based on accurate beam modelling for homogeneous tumour tissue. More patients and other cancer sites are needed in a further study to validate the proposed method fully.

Advances in knowledge

With accurate beam modelling, the deep learning model can quickly generate the dose distribution for CyberKnife cases. This method accelerates the RT planning process, significantly improves its operational efficiency, and optimizes it.

Semi-automated segmentation methods of SSTR PET for dosimetry prediction in refractory meningioma patients treated by SSTR-targeted peptide receptor radionuclide therapy

OBJECTIVES

Tumor dosimetry with somatostatin receptor-targeted peptide receptor radionuclide therapy (SSTR-targeted PRRT) by 177Lu-DOTATATE may contribute to improved treatment monitoring of refractory meningioma. Accurate dosimetry requires reliable and reproducible pretherapeutic PET tumor segmentation which is not currently available. This study aims to propose semi-automated segmentation methods to determine metabolic tumor volume with pretherapeutic 68Ga-DOTATOC PET and evaluate SUVmean-derived values as predictive factors for tumor-absorbed dose.

METHODS

Thirty-nine meningioma lesions from twenty patients were analyzed. The ground truth PET and SPECT volumes (VolGT-PET and VolGT-SPECT) were computed from manual segmentations by five experienced nuclear physicians. SUV-related indexes were extracted from VolGT-PET and the semi-automated PET volumes providing the best Dice index with VolGT-PET (Volopt) across several methods: SUV absolute-value (2.3)-threshold, adaptative methods (Jentzen, Otsu, Contrast-based method), advanced gradient-based technique, and multiple relative thresholds (% of tumor SUVmax, hypophysis SUVmean, and meninges SUVpeak) with optimal threshold optimized. Tumor-absorbed doses were obtained from the VolGT-SPECT, corrected for partial volume effect, performed on a 360° whole-body CZT-camera at 24, 96, and 168 h after administration of 177Lu-DOTATATE.

RESULTS

Volopt was obtained from 1.7-fold meninges SUVpeak (Dice index 0.85 ± 0.07). SUVmean and total lesion uptake (SUVmeanxlesion volume) showed better correlations with tumor-absorbed doses than SUVmax when determined with the VolGT (respective Pearson correlation coefficients of 0.78, 0.67, and 0.56) or Volopt (0.64, 0.66, and 0.56).

CONCLUSION

Accurate definition of pretherapeutic PET volumes is justified since SUVmean-derived values provide the best tumor-absorbed dose predictions in refractory meningioma patients treated by 177Lu-DOTATATE. This study provides a semi-automated segmentation method of pretherapeutic 68Ga-DOTATOC PET volumes to achieve good reproducibility between physicians.

CLINICAL RELEVANCE STATEMENT

SUVmean-derived values from pretherapeutic 68Ga-DOTATOC PET are predictive of tumor-absorbed doses in refractory meningiomas treated by 177Lu-DOTATATE, justifying to accurately define pretherapeutic PET volumes. This study provides a semi-automated segmentation of 68Ga-DOTATOC PET images easily applicable in routine.

KEY POINTS

• SUVmean-derived values from pretherapeutic 68Ga-DOTATOC PET images provide the best predictive factors of tumor-absorbed doses related to 177Lu-DOTATATE PRRT in refractory meningioma. • A 1.7-fold meninges SUVpeak segmentation method used to determine metabolic tumor volume on pretherapeutic 68Ga-DOTATOC PET images of refractory meningioma treated by 177Lu-DOTATATE is as efficient as the currently routine manual segmentation method and limits inter- and intra-observer variabilities. • This semi-automated method for segmentation of refractory meningioma is easily applicable to routine practice and transferrable across PET centers.

An overview of artificial intelligence in medical physics and radiation oncology

Artificial intelligence (AI) is developing rapidly and has found widespread applications in medicine, especially radiotherapy. This paper provides a brief overview of AI applications in radiotherapy, and highlights the research directions of AI that can potentially make significant impacts and relevant ongoing research works in these directions. Challenging issues related to the clinical applications of AI, such as robustness and interpretability of AI models, are also discussed. The future research directions of AI in the field of medical physics and radiotherapy are highlighted.

CT-Only Radiotherapy: An Exploratory Study for Automatic Dose Prediction on Rectal Cancer Patients Via Deep Adversarial Network

Purpose

Current deep learning methods for dose prediction require manual delineations of planning target volume (PTV) and organs at risk (OARs) besides the original CT images. Perceiving the time cost of manual contour delineation, we expect to explore the feasibility of accelerating the radiotherapy planning by leveraging only the CT images to produce high-quality dose distribution maps while generating the contour information automatically.

Materials and Methods

We developed a generative adversarial network (GAN) with multi-task learning (MTL) strategy to produce accurate dose distribution maps without manually delineated contours. To balance the relative importance of each task (i.e., the primary dose prediction task and the auxiliary tumor segmentation task), a multi-task loss function was employed. Our model was trained, validated and evaluated on a cohort of 130 rectal cancer patients.

Results

Experimental results manifest the feasibility and improvements of our contour-free method. Compared to other mainstream methods (i.e., U-net, DeepLabV3+, DoseNet, and GAN), the proposed method produces the leading performance with statistically significant improvements by achieving the highest HI of 1.023 (3.27E-5) and the lowest prediction error with ΔD95 of 0.125 (0.035) and ΔDmean of 0.023 (4.19E-4), respectively. The DVH differences between the predicted dose and the ideal dose are subtle and the errors in the difference maps are minimal. In addition, we conducted the ablation study to validate the effectiveness of each module. Furthermore, the results of attention maps also prove that our CT-only prediction model is capable of paying attention to both the target tumor (i.e., high dose distribution area) and the surrounding healthy tissues (i.e., low dose distribution areas).

Conclusion

The proposed CT-only dose prediction framework is capable of producing acceptable dose maps and reducing the time and labor for manual delineation, thus having great clinical potential in providing accurate and accelerated radiotherapy. Code is available at https://github.com/joegit-code/DoseWithCT