Predictive gamma passing rate by dose uncertainty potential accumulation model

Unbiased evaluation of predicted gamma passing rate by an event-mixing technique

BACKGROUND

Predicting models of the gamma passing rate (GPR) have been studied to substitute the measurement-based gamma analysis. Since these studies used data from different radiotherapy systems comprising TPS, linear accelerator, and detector array, it has been difficult to compare the performances of the predicting models among institutions with different radiotherapy systems.

PURPOSE

We aimed to develop unbiased scoring methods to evaluate the performance of the models predicting the GPR, by introducing both best and worst limits for the performance of the GPR prediction.

METHODS

Two hundred head-and-neck VMAT plans were used to develop a framework. The GPRs were measured using the ArcCHECK device. The predicted GPR [p] was generated using a deep learning-based model [pDL ]. The predicting model was evaluated using four metrics: standard deviation (SD) [σ], Pearson's correlation coefficient (CC) [r], mean squared error (MSE) [s], and mean absolute error (MAE) [a]. The best limit [σ m ${\sigma _m}$ ,r m ${r_m}$ ,s m ${s_m}$ , anda m ${a_m}$ ] was estimated by measuring the SD of measured GPR [m] by shifting the device along the longitudinal direction to measure different sampling points. Mimicked best and worst p's [pbest and pworst ] were generated from pDL . The worst limit was defined such that m and p have no correlation [CC ∼ 0]. The worst limit [σMix , rMix , sMix , and aMix ] was generated using the event-mixing (EM) technique originally introduced in high-energy physics experiments. The range of σ, r, s, and a was defined to be[ σ m , σ Mix ] $[ {{\sigma _m},{\sigma _{{\mathrm{Mix}}}}} ]$ ,[ 0 , r m ] $[ {0,{r_m}} ]$ ,[ s m , s Mix ] $[ {{s_m},{s_{{\mathrm{Mix}}}}} ]$ , and[ a m , a Mix ] $[ {{a_m},{a_{{\mathrm{Mix}}}}} ]$ . The achievement score (AS) independently based on σ, r, s, and a were calculated for pDL , pbest and pworst . The probability that p fails the gamma analysis (alert frequency; AF) was estimated as a function ofσ d ${\sigma _d}$ values within the [σ m ${\sigma _m}$ , σMix ] range for the 3%/2 mm data with a 95% criterion.

RESULTS

SDs of the best limit were well reproduced byσ m = 0.531 100 - m ${\sigma _m} = \;0.531\sqrt {100 - m} $ . The EM technique successfully generated the( m , p ) $( {m,p} )$ pairs with no correlation. The AS using four metrics showed good agreement. This agreement indicates successful definitions of both best and worst limits, consistent definitions of the AS, and successful generations of mixed events. The AF for the DL-based model with the 3%/2 mm tolerance was 31.5% and 63.0% with CL's 99% and 99.9%, respectively.

CONCLUSION

We developed the AS to evaluate the predicting model of the GPR in an unbiased manner by excluding the effects of the precision of the radiotherapy system and the spreading of the GPR. The best and worst limits of the GPR prediction were successfully generated using the measured precision of the GPR and the EM technique, respectively. The AS andσ p ${\sigma _p}$ are expected to enable objective evaluation of the predicting model and setting exact achievement goal of precision for the predicted GPR.

Prediction of patient-specific quality assurance for volumetric modulated arc therapy using radiomics-based machine learning with dose distribution

PURPOSE

We sought to develop machine learning models to predict the results of patient-specific quality assurance (QA) for volumetric modulated arc therapy (VMAT), which were represented by several dose-evaluation metrics-including the gamma passing rates (GPRs)-and criteria based on the radiomic features of 3D dose distribution in a phantom.

METHODS

A total of 4,250 radiomic features of 3D dose distribution in a cylindrical dummy phantom for 140 arcs from 106 clinical VMAT plans were extracted. We obtained the following dose-evaluation metrics: GPRs with global and local normalization, the dose difference (DD) in 1% and 2% passing rates (DD1% and DD2%) for 10% and 50% dose threshold, and the distance-to-agreement in 1-mm and 2-mm passing rates (DTA1 mm and DTA2 mm) for 0.5%/mm and 1.0%.mm dose gradient threshold determined by measurement using a diode array in patient-specific QA. The machine learning regression models for predicting the values of the dose-evaluation metrics using the radiomic features were developed based on the elastic net (EN) and extra trees (ET) models. The feature selection and tuning of hyperparameters were performed with nested cross-validation in which four-fold cross-validation is used within the inner loop, and the performance of each model was evaluated in terms of the root mean square error (RMSE), the mean absolute error (MAE), and Spearman's rank correlation coefficient.

RESULTS

The RMSE and MAE for the developed machine learning models ranged from <1% to nearly <10% depending on the dose-evaluation metric, the criteria, and dose and dose gradient thresholds used for both machine learning models. It was advantageous to focus on high dose region for predicating global GPR, DDs, and DTAs. For certain metrics and criteria, it was possible to create models applicable for patients' heterogeneity by training only with dose distributions in phantom.

CONCLUSIONS

The developed machine learning models showed high performance for predicting dose-evaluation metrics especially for high dose region depending on the metric and criteria. Our results demonstrate that the radiomic features of dose distribution can be considered good indicators of the plan complexity and useful in predicting measured dose evaluation metrics.

Impact of ovary-sparing treatment planning on plan quality, treatment time and gamma passing rates in intensity-modulated radiotherapy for stage I/II cervical cancer

BACKGROUND

This study aimed to investigate the impact of ovary-sparing intensity-modulated radiotherapy (IMRT) on plan quality, treatment time, and gamma passing rates for stage I/II cervical cancer patients.

METHODS

Fifteen stage I/II cervical cancer patients were retrospectively enrolled, and a pair of clinically suitable IMRT plans were designed for each patient, with (Group A) and without (Group B) ovary-sparing. Plan factors affecting plan quality, treatment time, and gamma passing rates, including the number of segments, monitor units, percentage of small-area segments (field area < 20 cm2), and percentage of small-MU segments (MU < 10), were compared and statistically analyzed. Key plan quality indicators, including ovarian dose, target dose coverage (D98%, D95%, D50%, D2%), conformity index, and homogeneity index, were evaluated and statistically assessed. Treatment time and gamma passing rates collected by IBA MatriXX were also compared.

RESULTS

The median ovarian dose in Group A and Group B was 7.61 Gy (range 6.71-8.51 Gy) and 38.52 Gy (range 29.84-43.82 Gy), respectively. Except for monitor units, all other plan factors were significantly lower in Group A than in Group B (all P < .05). Correlation coefficients between plan factors, treatment time, and gamma passing rates that were statistically different were all negative. Both Groups of plans met the prescription requirement (D95% ≥ 45.00 Gy) for clinical treatment. D98% was smaller for Group A than for Group B (P < .05); D50% and D2% were larger for Group A than for Group B (P < .05, P < .05). Group A plans had worse conformity index and homogeneity index than Group B plans (P < .05, P < .05). Treatment time did not differ significantly (P > .05). Gamma passing rates in Group A were higher than in Group B with the criteria of 2%/3 mm (P < .05) and 3%/2 mm (P < .05).

CONCLUSION

Despite the slightly decreased quality of the treatment plans, the ovary-sparing IMRT plans exhibited several advantages including lower ovarian dose and plan complexity, improved gamma passing rates, and a negligible impact on treatment time.

A synthesized gamma distribution-based patient-specific VMAT QA using a generative adversarial network

BACKGROUND

Artificial intelligence (AI)-based gamma passing rate (GPR) prediction has been proposed as a time-efficient virtual patient-specific QA method for the delivery of volumetric modulation arc therapy (VMAT). However, there is a limitation that the GPR value loses the locational information of dose accuracy.

PURPOSE

The objective was to predict the failing points in the gamma distribution and the GPR using a synthesized gamma distribution of VMAT QA with a deep convolutional generative adversarial network (GAN).

METHODS

The fluence maps of 270 VMAT beams for prostate cancer were measured using an electronic portal imaging device and analyzed using gamma evaluation with 3%/2-mm, 2%/1-mm, 1%/1-mm, and 1%/0.5-mm tolerances. The 270 gamma distributions were divided into two datasets: 240 training datasets for creating a model and 30 test datasets for evaluation. The image prediction network for the fluence maps calculated by the treatment planning system (TPS) to the gamma distributions was created using a GAN. The sensitivity, specificity, and accuracy of detecting failing points were evaluated using measured and synthesized gamma distributions. In addition, the difference between measured GPR (mGPR) and predicted GPR (pGPR) values calculated from the synthesized gamma distributions was evaluated.

RESULTS

The root mean squared errors between mGPR and pGPR were 1.0%, 2.1%, 3.5%, and 3.6% for the 3%/2-mm, 2%/1-mm, 1%/1-mm, and 1%/0.5-mm tolerances, respectively. The accuracies for detecting failing points were 98.9%, 96.9%, 94.7%, and 93.7% for 3%/2-mm, 2%/1-mm, 1%/1-mm, and 1%/0.5-mm tolerances, respectively. The sensitivity and specificity were the highest for 1%/0.5-mm and 3%/2-mm tolerances, which were 82.7% and 99.6%, respectively.

CONCLUSIONS

We developed a novel system using a GAN to generate a synthesized gamma distribution-based patient-specific VMAT QA. The system is promising from the point of view of quality assurance in radiotherapy because it shows high performance and can detect failing points.

Predictive gamma passing rate of 3D detector array-based volumetric modulated arc therapy quality assurance for prostate cancer via deep learning

To predict the gamma passing rate (GPR) of the three-dimensional (3D) detector array-based volumetric modulated arc therapy (VMAT) quality assurance (QA) for prostate cancer using a convolutional neural network (CNN) with the 3D dose distribution. One hundred thirty-five VMAT plans for prostate cancer were selected: 110 plans were used for training and validation, and 25 plans were used for testing. Verification plans were measured using a helical 3D diode array (ArcCHECK). The dose distribution on the detector element plane of these verification plans was used as input data for the CNN model. The measured GPR (mGPR) values were used as the training data. The CNN model comprises eighteen layers and predicted GPR (pGPR) values. The mGPR and pGPR values were compared, and a cumulative frequency histogram of the prediction error was created to clarify the prediction error tendency. The correlation coefficients of pGPR and mGPR were 0.67, 0.69, 0.66, and 0.73 for 3%/3-mm, 3%/2-mm, 2%/3-mm, and 2%/2-mm gamma criteria, respectively. The respective mean±standard deviations of pGPR-mGPR were -0.87±2.18%, -0.65±2.93%, -0.44±2.53%, and -0.71±3.33%. The probabilities of false positive error cases (pGPR < mGPR) were 72%, 60%, 68%, and 56% for each gamma criterion. We developed a deep learning-based prediction model of the 3D detector array-based VMAT QA for prostate cancer, and evaluated the accuracy and tendency of prediction GPR. This model can provide a proactive estimation for the results of the patient-specific QA before the verification measurement.

Evaluation of prediction and classification performances in different machine learning models for patient-specific quality assurance of head-and-neck VMAT plans

PURPOSE

The purpose of this study is to evaluate the prediction and classification performances of the gamma passing rate (GPR) for different machine learning models and to select the best model for achieving machine learning-based patient-specific quality assurance (PSQA).

METHODS

The measurement verification of 356 head-and-neck volumetric modulated arc therapy plans was performed using a diode array phantom (Delta4 Phantom), and GPR values at 2%/2 mm with global normalization and 3%/2 mm with local normalization were calculated. Machine learning models, including ridge regression (RIDGE), random forest (RF), support vector regression (SVR), and stacked generalization (STACKING), were used to predict the GPR. Each machine learning model was trained using 260 plans, and the prediction accuracy was evaluated using the remaining 96 plans. The prediction error between the measured and predicted GPR was evaluated. For the classification evaluation, the lower control limit for the measured GPR and lower control limit for predicted GPR (LCL_p ) was defined to identify whether the GPR values represent a "pass" or a "fail." LCL_p values with 99% and 99.9% confidence levels were calculated as the upper prediction limits for the GPR estimated from the linear regression between the measured and predicted GPR.

RESULTS

There was an overestimation trend of the low measured GPR. The maximum prediction errors for RIDGE, RF, SVR, and STACKING were 3.2%, 2.9%, 2.3%, and 2.2% at the global 2%/2 mm and 6.3%, 6.6%, 6.1%, and 5.5% at the local 3%/2 mm, respectively. In the global 2%/2 mm, the sensitivity was 100% for all the machine learning models except RIDGE when using 99% LCL_p . The specificity was 76.1% for RIDGE, RF, and SVR and 66.3% for STACKING; however, the specificity decreased dramatically when 99.9% LCL_p was used. In the local 3%/2 mm, however, only STACKING showed 100% sensitivity when using 99% LCL_p . The decrease in the specificity using 99.9% LCL_p was smaller than that in the global 2%/2 mm, and the specificity for RIDGE, RF, SVR, and STACKING was 61.3%, 61.3%, 72.0%, and 66.8%, respectively.

CONCLUSIONS

STACKING had better prediction accuracy for low GPR values than other machine learning models. Applying LCL_p to a regression model enabled the consistent evaluation of quantitative and qualitative GPR predictions. Adjusting the confidence level of the LCL_p helped improve the balance between the sensitivity and specificity. We suggest that STACKING can assist the safe and efficient operation of PSQA.

A machine learning-based framework for delivery error prediction in proton pencil beam scanning using irradiation log-files

PURPOSE

This study aims to investigate the use of machine learning models for delivery error prediction in proton pencil beam scanning (PBS) delivery.

METHODS

A dataset of planned and delivered PBS spot parameters was generated from a set of 20 prostate patient treatments. Planned spot parameters (spot position, MU and energy) were extracted from the treatment planning system (TPS) for each beam. Delivered spot parameters were extracted from irradiation log-files for each beam delivery following treatment. The dataset was used as a training dataset for three machine learning models which were trained to predict delivered spot parameters based on planned parameters. K-fold cross validation was employed for hyper-parameter tuning and model selection where the mean absolute error (MAE) was used as the model evaluation metric. The model with lowest MAE was then selected to generate a predicted dose distribution for a test prostate patient within a commercial TPS.

RESULTS

Analysis of the spot position delivery error between planned and delivered values resulted in standard deviations of 0.39 mm and 0.44 mm for x and y spot positions respectively. Prediction error standard deviation values of spot positions using the selected model were 0.22 mm and 0.11 mm for x and y spot positions respectively. Finally, a three-way comparison of dose distributions and DVH values for select OARs indicates that the random-forest-predicted dose distribution within the test prostate patient was in closer agreement to the delivered dose distribution than the planned distribution.

CONCLUSIONS

PBS delivery error can be accurately predicted using machine learning techniques.

A technique for quantifying the sensitivity of dosimetric tool gamma with 2D detector array in pretreatment IMRT plans by segment deletion method

OBJECTIVES

Motivation of this study is to check the sensitivity of dosimetric tool gamma with 2D detector array combination when unexpected errors occur while transferring intensity-modulated radiation therapy treatment plans from planning system to treatment unit.

METHODS

This study consists of 17 head and neck cancer patient's treatment plans. Nine types of verification plans are created for all 17 clinically approved treatment plans by consecutively deleting different segments (up to eight) one by one from each field of the plan. Decrement factor (χ) is introduced in our study which illustrated the degree of decay of gamma passing rate when intentional errors are introduced. We analyzed the data by two different methods-one without selecting the region of interest (ROI) in dose distributions and the other by selecting the region of interest.

RESULTS

By linear regression, the absolute value of slopes is 0.025, 0.024 and 0.015 without ROI and 0.030, 0.027 and 0.015 with ROI for 2%/2 mm, 3%/3 mm and 5%/5 mm criteria, respectively. The higher absolute value of the fitted slope indicates the higher sensitivity of this method to identify erroneous plan in treatment unit. The threshold value for 2%/2 mm equivalent to 95% passing criteria in 3%/3 mm used in clinical practice is obtained as 83.44%.

CONCLUSIONS

The 2D detector array with dosimetric tool gamma is less sensitive in detecting errors when unprecedented errors of segment deletion occur within the treatment plans.

Study on the ability of 3D gamma analysis and bio-mathematical model in detecting dose changes caused by dose-calculation-grid-size (DCGS)

Objective

To explore the efficacy and sensitivity of 3D gamma analysis and bio-mathematical model for cervical cancer in detecting dose changes caused by dose-calculation-grid-size (DCGS).

Methods

17 patients’ plans for cervical cancer were enrolled (Pinnacle TPS, VMAT), and the DCGS was changed from 2.0 mm to 5.0 mm to calculate the planned dose respectively. The dose distribution calculated by DCGS = 2.0 mm as the “reference” data set (RDS), the dose distribution calculated by the rest DCGS as the“measurement”data set (MDS), the 3D gamma passing rates and the (N) TCPs of the all structures under different DCGS were obtained, and then analyze the ability of 3D gamma analysis and (N) TCP model in detecting dose changes and what factors affect this ability.

Results

The effect of DCGS on planned dose was obvious. When the gamma standard was 1.0 mm, 1.0 and 10.0%, the difference of the results of the DCGS on dose-effect could be detected by 3D gamma analysis (all p value < 0.05). With the decline of the standard, 3D gamma analysis’ ability to detect this difference shows weaker. When the standard was 1.0 mm, 3.0 and 10.0%, the p value of > 0.05 accounted for the majority. With DCGS = 2.0 mm being RDS, ∆gamma-passing-rate presented the same trend with ∆(N) TCPs of all structures except for the femurs only when the 1.0 mm, 1.0 and 10.0% standards were adopted for the 3D gamma analysis.

Conclusions

The 3D gamma analysis and bio-mathematical model can be used to analyze the effect of DCGS on the planned dose. For comparison, the former’s detection ability has a lot to do with the designed standard, and the latter’s capability is related to the parameters and calculated accuracy instrinsically.

Prediction of dosimetric accuracy for VMAT plans using plan complexity parameters via machine learning

PURPOSE

The dosimetric accuracies of volumetric modulated arc therapy (VMAT) plans were predicted using plan complexity parameters via machine learning.

METHODS

The dataset consisted of 600 cases of clinical VMAT plans from a single institution. The predictor variables (n = 28) for each plan included complexity parameters, machine type, and photon beam energy. Dosimetric measurements were performed using a helical diode array (ArcCHECK), and the dosimetric accuracy of the passing rates for a 5% dose difference (DD5%) and gamma index of 3%/3 mm (γ3%/3 mm) were predicted using three machine learning models: regression tree analysis (RTA), multiple regression analysis (MRA), and neural networks (NNs). First, the prediction models were applied to 500 cases of the VMAT plans. Then, the dosimetric accuracy was predicted using each model for the remaining 100 cases (evaluation dataset). The error between the predicted and measured passing rates was evaluated.

RESULTS

For the 600 cases, the mean ± standard deviation of the measured passing rates was 92.3% ± 9.1% and 96.8% ± 3.1% for DD5% and γ3%/3 mm, respectively. For the evaluation dataset, the mean ± standard deviation of the prediction errors for DD5% and γ3%/3 mm was 0.5% ± 3.0% and 0.6% ± 2.4% for RTA, 0.0% ± 2.9% and 0.5% ± 2.4% for MRA, and -0.2% ± 2.7% and -0.2% ± 2.1% for NN, respectively.

CONCLUSIONS

NNs performed slightly better than RTA and MRA in terms of prediction error. These findings may contribute to increasing the efficiency of patient-specific quality-assurance procedures.