Using machine learning to predict gamma passing rate in volumetric-modulated arc therapy treatment plans

Unbiased evaluation of predicted gamma passing rate by an event-mixing technique

BACKGROUND

Predicting models of the gamma passing rate (GPR) have been studied to substitute the measurement-based gamma analysis. Since these studies used data from different radiotherapy systems comprising TPS, linear accelerator, and detector array, it has been difficult to compare the performances of the predicting models among institutions with different radiotherapy systems.

PURPOSE

We aimed to develop unbiased scoring methods to evaluate the performance of the models predicting the GPR, by introducing both best and worst limits for the performance of the GPR prediction.

METHODS

Two hundred head-and-neck VMAT plans were used to develop a framework. The GPRs were measured using the ArcCHECK device. The predicted GPR [p] was generated using a deep learning-based model [pDL ]. The predicting model was evaluated using four metrics: standard deviation (SD) [σ], Pearson's correlation coefficient (CC) [r], mean squared error (MSE) [s], and mean absolute error (MAE) [a]. The best limit [σ m ${\sigma _m}$ ,r m ${r_m}$ ,s m ${s_m}$ , anda m ${a_m}$ ] was estimated by measuring the SD of measured GPR [m] by shifting the device along the longitudinal direction to measure different sampling points. Mimicked best and worst p's [pbest and pworst ] were generated from pDL . The worst limit was defined such that m and p have no correlation [CC ∼ 0]. The worst limit [σMix , rMix , sMix , and aMix ] was generated using the event-mixing (EM) technique originally introduced in high-energy physics experiments. The range of σ, r, s, and a was defined to be[ σ m , σ Mix ] $[ {{\sigma _m},{\sigma _{{\mathrm{Mix}}}}} ]$ ,[ 0 , r m ] $[ {0,{r_m}} ]$ ,[ s m , s Mix ] $[ {{s_m},{s_{{\mathrm{Mix}}}}} ]$ , and[ a m , a Mix ] $[ {{a_m},{a_{{\mathrm{Mix}}}}} ]$ . The achievement score (AS) independently based on σ, r, s, and a were calculated for pDL , pbest and pworst . The probability that p fails the gamma analysis (alert frequency; AF) was estimated as a function ofσ d ${\sigma _d}$ values within the [σ m ${\sigma _m}$ , σMix ] range for the 3%/2 mm data with a 95% criterion.

RESULTS

SDs of the best limit were well reproduced byσ m = 0.531 100 - m ${\sigma _m} = \;0.531\sqrt {100 - m} $ . The EM technique successfully generated the( m , p ) $( {m,p} )$ pairs with no correlation. The AS using four metrics showed good agreement. This agreement indicates successful definitions of both best and worst limits, consistent definitions of the AS, and successful generations of mixed events. The AF for the DL-based model with the 3%/2 mm tolerance was 31.5% and 63.0% with CL's 99% and 99.9%, respectively.

CONCLUSION

We developed the AS to evaluate the predicting model of the GPR in an unbiased manner by excluding the effects of the precision of the radiotherapy system and the spreading of the GPR. The best and worst limits of the GPR prediction were successfully generated using the measured precision of the GPR and the EM technique, respectively. The AS andσ p ${\sigma _p}$ are expected to enable objective evaluation of the predicting model and setting exact achievement goal of precision for the predicted GPR.

Prediction of patient-specific quality assurance for volumetric modulated arc therapy using radiomics-based machine learning with dose distribution

PURPOSE

We sought to develop machine learning models to predict the results of patient-specific quality assurance (QA) for volumetric modulated arc therapy (VMAT), which were represented by several dose-evaluation metrics-including the gamma passing rates (GPRs)-and criteria based on the radiomic features of 3D dose distribution in a phantom.

METHODS

A total of 4,250 radiomic features of 3D dose distribution in a cylindrical dummy phantom for 140 arcs from 106 clinical VMAT plans were extracted. We obtained the following dose-evaluation metrics: GPRs with global and local normalization, the dose difference (DD) in 1% and 2% passing rates (DD1% and DD2%) for 10% and 50% dose threshold, and the distance-to-agreement in 1-mm and 2-mm passing rates (DTA1 mm and DTA2 mm) for 0.5%/mm and 1.0%.mm dose gradient threshold determined by measurement using a diode array in patient-specific QA. The machine learning regression models for predicting the values of the dose-evaluation metrics using the radiomic features were developed based on the elastic net (EN) and extra trees (ET) models. The feature selection and tuning of hyperparameters were performed with nested cross-validation in which four-fold cross-validation is used within the inner loop, and the performance of each model was evaluated in terms of the root mean square error (RMSE), the mean absolute error (MAE), and Spearman's rank correlation coefficient.

RESULTS

The RMSE and MAE for the developed machine learning models ranged from <1% to nearly <10% depending on the dose-evaluation metric, the criteria, and dose and dose gradient thresholds used for both machine learning models. It was advantageous to focus on high dose region for predicating global GPR, DDs, and DTAs. For certain metrics and criteria, it was possible to create models applicable for patients' heterogeneity by training only with dose distributions in phantom.

CONCLUSIONS

The developed machine learning models showed high performance for predicting dose-evaluation metrics especially for high dose region depending on the metric and criteria. Our results demonstrate that the radiomic features of dose distribution can be considered good indicators of the plan complexity and useful in predicting measured dose evaluation metrics.