Error detection and classification in patient-specific IMRT QA with dual neural networks

A StarGAN and transformer-based hybrid classification-regression model for multi-institution VMAT patient-specific quality assurance

BACKGROUND

The field of artificial intelligence (AI)-based patient-specific quality assurance (PSQA) for volumetric modulated arc therapy (VMAT) faces challenges in terms of developing general models across institutions due to the prevalence of multi-institution data collection and multivariate heterogeneity. Building a general model that is capable of handling diverse multi-institution data is critical for enabling large-scale integration and analysis.

PURPOSE

This study aims to develop a star generative adversarial network (StarGAN) and transformer-based hybrid classification-regression PSQA framework to address unification of heterogeneous data from different institutions.

METHODS

A StarGAN and transformer-based hybrid classification-regression model was developed as a general PSQA framework to predict gamma passing rates (GPRs) and classify quality assurance (QA) results as "Pass" or "Fail" at multiple institutions. A total of 1815 VMAT plans were collected from eight institutions to develop the general PSQA framework and perform clinical commissioning and implementation. Among them, 20 independent clinical plans from each of eight institutions, for a total of 160 plans, were used for the clinical commissioning, and 205 new clinical plans from eight institutions were used for clinical implementation.

RESULTS

For the 3%/3, 3%/2, and 2%/2 mm gamma criteria, the sensitivity of the proposed PSQA framework with pretraining was 90.13%, 92.03%, and 95.84%, respectively, while the specificity was 76.01%, 76.12%, and 85.34%, respectively. The mean absolute errors (MAEs) of the proposed PSQA framework with pretraining were 1.36%, 2.37%, and 3.96%, respectively, while the root-mean-square errors (RMSEs) were 2.31%, 3.89%, and 5.17%, respectively. The results demonstrated visible improvement at multiple institutions. For clinical commissioning, the deviations between the predicted and measured results were all within 3% for 3%/3 and 3%/2 mm at eight institutions. For clinical implementation, all failure plans were correctly identified by the proposed PSQA framework.

CONCLUSIONS

The general PSQA framework enables diverse clinical data sources to be handled to achieve enhanced model performance and generalizability, and provides a solution to the unification of heterogeneous data from different institutions to construct robust QA models. This approach can be clinically deployed for VMAT QA.

In vivo EPID-based daily treatment error identification for volumetric-modulated arc therapy in head and neck cancers with a hierarchical convolutional neural network: a feasibility study

We proposed a deep learning approach to classify various error types in daily VMAT treatment of head and neck cancer patients based on EPID dosimetry, which could provide additional information to support clinical decisions for adaptive planning. 146 arcs from 42 head and neck patients were analyzed. Anatomical changes and setup errors were simulated in 17,820 EPID images of 99 arcs obtained from 30 patients using in-house software for model training, validation, and testing. Subsequently, 141 clinical EPID images from 47 arcs belonging to the remaining 12 patients were utilized for clinical testing. The hierarchical convolutional neural network (HCNN) model was trained to classify error types and magnitudes using EPID dose difference maps. Gamma analysis with 3%/2 mm (dose difference/distance to agreement) criteria was also performed. The F1 score, a combination of precision and recall, was utilized to evaluate the performance of the HCNN model and gamma analysis. The adaptive fractioned doses were calculated to verify the HCNN classification results. For error type identification, the overall F1 score of the HCNN model was 0.99 and 0.91 for primary type and subtype identification, respectively. For error magnitude identification, the overall F1 score in the simulation dataset was 0.96 and 0.70 for the HCNN model and gamma analysis, respectively; while the overall F1 score in the clinical dataset was 0.79 and 0.20 for the HCNN model and gamma analysis, respectively. The HCNN model-based EPID dosimetry can identify changes in patient transmission doses and distinguish the treatment error category, which could potentially provide information for head and neck cancer treatment adaption.

Turning the attention to time-resolved EPID-images: treatment error classification with transformer multiple instance learning

Objective.The aim of this work was to develop a novel artificial intelligence-assistedin vivodosimetry method using time-resolved (TR) dose verification data to improve quality of external beam radiotherapy.Approach. Although threshold classification methods are commonly used in error classification, they may lead to missing errors due to the loss of information resulting from the compression of multi-dimensional electronic portal imaging device (EPID) data into one or a few numbers. Recent research has investigated the classification of errors on time-integrated (TI)in vivoEPID images, with convolutional neural networks showing promise. However, it has been observed previously that TI approaches may cancel out the error presence onγ-maps during dynamic treatments. To address this limitation, simulated TRγ-maps for each volumetric modulated arc radiotherapy angle were used to detect treatment errors caused by complex patient geometries and beam arrangements. Typically, such images can be interpreted as a set of segments where only set class labels are provided. Inspired by recent weakly supervised approaches on histopathology images, we implemented a transformer based multiple instance learning approach and utilized transfer learning from TI to TRγ-maps.Main results. The proposed algorithm performed well on classification of error type and error magnitude. The accuracy in the test set was up to 0.94 and 0.81 for 11 (error type) and 22 (error magnitude) classes of treatment errors, respectively.Significance. TR dose distributions can enhance treatment delivery decision-making, however manual data analysis is nearly impossible due to the complexity and quantity of this data. Our proposed model efficiently handles data complexity, substantially improving treatment error classification compared to models that leverage TI data.

Error detection for radiotherapy planning validation based on deep learning networks

BACKGROUND

Quality assurance (QA) of patient-specific treatment plans for intensity-modulated radiation therapy (IMRT) and volumetric modulated arc therapy (VMAT) necessitates prior validation. However, the standard methodology exhibits deficiencies and lacks sensitivity in the analysis of positional dose distribution data, leading to difficulties in accurately identifying reasons for plan verification failure. This issue complicates and impedes the efficiency of QA tasks.

PURPOSE

The primary aim of this research is to utilize deep learning algorithms for the extraction of 3D dose distribution maps and the creation of a predictive model for error classification across multiple machine models, treatment methodologies, and tumor locations.

METHOD

We devised five categories of validation plans (normal, gantry error, collimator error, couch error, and dose error), conforming to tolerance limits of different accuracy levels and employing 3D dose distribution data from a sample of 94 tumor patients. A CNN model was then constructed to predict the diverse error types, with predictions compared against the gamma pass rate (GPR) standard employing distinct thresholds (3%, 3 mm; 3%, 2 mm; 2%, 2 mm) to evaluate the model's performance. Furthermore, we appraised the model's robustness by assessing its functionality across diverse accelerators.

RESULTS

The accuracy, precision, recall, and F1 scores of CNN model performance were 0.907, 0.925, 0.907, and 0.908, respectively. Meanwhile, the performance on another device is 0.900, 0.918, 0.900, and 0.898. In addition, compared to the GPR method, the CNN model achieved better results in predicting different types of errors.

CONCLUSION

When juxtaposed with the GPR methodology, the CNN model exhibits superior predictive capability for classification in the validation of the radiation therapy plan on different devices. By using this model, the plan validation failures can be detected more rapidly and efficiently, minimizing the time required for QA tasks and serving as a valuable adjunct to overcome the constraints of the GPR method.

Applications of artificial intelligence for machine- and patient-specific quality assurance in radiation therapy: current status and future directions

Machine- and patient-specific quality assurance (QA) is essential to ensure the safety and accuracy of radiotherapy. QA methods have become complex, especially in high-precision radiotherapy such as intensity-modulated radiation therapy (IMRT) and volumetric modulated arc therapy (VMAT), and various recommendations have been reported by AAPM Task Groups. With the widespread use of IMRT and VMAT, there is an emerging demand for increased operational efficiency. Artificial intelligence (AI) technology is quickly growing in various fields owing to advancements in computers and technology. In the radiotherapy treatment process, AI has led to the development of various techniques for automated segmentation and planning, thereby significantly enhancing treatment efficiency. Many new applications using AI have been reported for machine- and patient-specific QA, such as predicting machine beam data or gamma passing rates for IMRT or VMAT plans. Additionally, these applied technologies are being developed for multicenter studies. In the current review article, AI application techniques in machine- and patient-specific QA have been organized and future directions are discussed. This review presents the learning process and the latest knowledge on machine- and patient-specific QA. Moreover, it contributes to the understanding of the current status and discusses the future directions of machine- and patient-specific QA.

Deep learning-based tools to distinguish plan-specific from generic deviations in EPID-based in vivo dosimetry

BACKGROUND

Dose distributions calculated with electronic portal imaging device (EPID)-based in vivo dosimetry (EIVD) differ from planned dose distributions due to generic and plan-specific deviations. Generic deviations are characteristic to a class of plans. Examples include limitations in EIVD dose reconstruction, inaccuracies in treatment planning system (TPS) calculations and systematic machine deviations. Plan-specific deviations have an unpredictable character. Examples include discrepancies between the patient model used for dose calculation and the patient position or anatomy during delivery, random machine deviations, and data transfer, human or software errors. During the inspection work performed with traditional γ-evaluation statistical methods: (i) generic deviations raise alerts that need to be inspected but that rarely lead to action as their root cause is usually understood and (ii) the detection of relevant plan-specific deviations may be hindered by the presence of generic deviations.

PURPOSE

To investigate whether deep learning-based tools can help in identifying γ-alerts raised by generic deviations and in improving the detectability of plan-specific deviations.

METHODS

A 3D U-Net was trained as an autoencoder to reconstruct underlying patterns of generic deviations in γ-distributions. The network was trained for four treatment disease sites differently affected by generic deviations: volumetric modulated arc therapy (VMAT) lung (no known deviations), VMAT prostate (TPS inaccuracies), VMAT head-and-neck (EIVD limitations) and intensity modulated radiation therapy (IMRT) breast (large EIVD limitations). The network was trained with virtual non-transit γ-distributions: 60 train/10 validation for the VMAT sites and 30 train/10 validation for IMRT breast. It was hypothesized that in vivo γ-distributions obtained in the presence of plan-specific deviations would differ from those seen during training. For each disease site, the sensitivity of γ-analysis and the network to detect (synthetically introduced) patient-related deviations was compared by receiver operator characteristic analysis. The investigated deviations were patient positioning errors, weight gain or loss, and tumor volume changes. The clinical relevance was illustrated qualitatively with 793 in vivo clinical cases (141 lung, 136 head-and-neck, 209 prostate and 307 breast).

RESULTS

Error detectability of patient-related deviations was better with the network than with γ-analysis. The average area under the curve values over all sites were 0.86 ± 0.12(1SD) and 0.69 ± 0.25(1SD), respectively. Regarding in vivo clinical results, the percentage of cases differently classified by γ-analysis and the network was 1%, 19%, 18% and 64% for lung, head-and-neck, prostate, and breast, respectively. In head-and-neck and breast cases, 45 γ-only alerts were examined, of which 43 were attributed to EPID dose reconstruction limitations. For prostate, all 15 investigated γ-only alerts were due to known TPS inaccuracies. All 59 investigated network alerts were explained by either patient-related deviations or EPID acquisition incidents. Some patient-related deviations detected by the network were not detected by γ-analysis.

CONCLUSIONS

Deep learning-based tools trained to reconstruct underlying patterns of generic deviations in γ-distributions can be used to (i) automatically identify false positives within the set of γ-alerts and (ii) improve the detection of plan-specific deviations, hence minimizing the likelihood of false negatives. The presented method provides clear additional value to the γ-alert management process for large scale EIVD systems.

A 3D transfer learning approach for identifying multiple simultaneous errors during radiotherapy

Objective. Deep learning models, such as convolutional neural networks (CNNs), can take full dose comparison images as input and have shown promising results for error identification during treatment. Clinically, complex scenarios should be considered, with the risk of multiple anatomical and/or mechanical errors occurring simultaneously during treatment. The purpose of this study was to evaluate the capability of CNN-based error identification in this more complex scenario.Approach. For 40 lung cancer patients, clinically realistic ranges of combinations of various treatment errors within treatment plans and/or computed tomography (CT) images were simulated. Modified CT images and treatment plans were used to predict 2580 3D dose distributions, which were compared to dose distributions without errors using various gamma analysis criteria and relative dose difference as dose comparison methods. A 3D CNN capable of multilabel classification was trained to identify treatment errors at two classification levels, using dose comparison volumes as input: Level 1 (main error type, e.g. anatomical change, mechanical error) and Level 2 (error subtype, e.g. tumor regression, patient rotation). For training the CNNs, a transfer learning approach was employed. An ensemble model was also evaluated, which consisted of three separate CNNs each taking a region of interest of the dose comparison volume as input. Model performance was evaluated by calculating sample F1-scores for training and validation sets.Main results. The model had high F1-scores for Level 1 classification, but performance for Level 2 was lower, and overfitting became more apparent. Using relative dose difference instead of gamma volumes as input improved performance for Level 2 classification, whereas using an ensemble model additionally reduced overfitting. The models obtained F1-scores of 0.86 and 0.62 on an independent test set for Level 1 and Level 2, respectively.Significance. This study shows that it is possible to identify multiple errors occurring simultaneously in 3D dose verification data.

Predicting the error magnitude in patient-specific QA during radiotherapy based on ResNet

BACKGROUND

The error magnitude is closely related to patient-specific dosimetry and plays an important role in evaluating the delivery of the radiotherapy plan in QA. No previous study has investigated the feasibility of deep learning to predict error magnitude.

OBJECTIVE

The purpose of this study was to predict the error magnitude of different delivery error types in radiotherapy based on ResNet.

METHODS

A total of 34 chest cancer plans (172 fields) of intensity-modulated radiation therapy (IMRT) from Eclipse were selected, of which 30 plans (151 fields) were used for model training and validation, and 4 plans including 21 fields were used for external testing. The collimator misalignment (COLL), monitor unit variation (MU), random multi-leaf collimator shift (MLCR), and systematic MLC shift (MLCS) were introduced. These dose distributions of portal dose predictions for the original plans were defined as the reference dose distribution (RDD), while those for the error-introduced plans were defined as the error-introduced dose distribution (EDD). Different inputs were used in the ResNet for predicting the error magnitude.

RESULTS

In the test set, the accuracy of error type prediction based on the dose difference, gamma distribution, and RDD + EDD was 98.36%, 98.91%, and 100%, respectively; the root mean squared error (RMSE) was 1.45-1.54, 0.58-0.90, 0.32-0.36, and 0.15-0.24; the mean absolute error (MAE) was 1.06-1.18, 0.32-0.78, 0.25-0.27, and 0.11-0.18, respectively, for COLL, MU, MLCR and MLCS.

CONCLUSIONS

In this study, error magnitude prediction models with dose difference, gamma distribution, and RDD + EDD are established based on ResNet. The accurate prediction of the error magnitude under different error types can provide reference for error analysis in patient-specific QA.

Deep learning-based detection and classification of multi-leaf collimator modeling errors in volumetric modulated radiation therapy

PURPOSE

The purpose of this study was to create and evaluate deep learning-based models to detect and classify errors of multi-leaf collimator (MLC) modeling parameters in volumetric modulated radiation therapy (VMAT), namely the transmission factor (TF) and the dosimetric leaf gap (DLG).

METHODS

A total of 33 clinical VMAT plans for prostate and head-and-neck cancer were used, assuming a cylindrical and homogeneous phantom, and error plans were created by altering the original value of the TF and the DLG by ± 10, 20, and 30% in the treatment planning system (TPS). The Gaussian filters ofσ = 0.5 $\sigma = 0.5$ and 1.0 were applied to the planar dose maps of the error-free plan to mimic the measurement dose map, and thus dose difference maps between the error-free and error plans were obtained. We evaluated 3 deep learning-based models, created to perform the following detections/classifications: (1) error-free versus TF error, (2) error-free versus DLG error, and (3) TF versus DLG error. Models to classify the sign of the errors were also created and evaluated. A gamma analysis was performed for comparison.

RESULTS

The detection and classification of TF and DLG error were feasible forσ = 0.5 $\sigma = 0.5$ ; however, a considerable reduction of accuracy was observed forσ = 1.0 $\sigma = 1.0$ depending on the magnitude of error and treatment site. The sign of errors was detectable by the specifically trained models forσ = 0.5 $\sigma = 0.5$ and 1.0. The gamma analysis could not detect errors.

CONCLUSIONS

We demonstrated that the deep learning-based models could feasibly detect and classify TF and DLG errors in VMAT dose distributions, depending on the magnitude of the error, treatment site, and the degree of mimicked measurement doses.

Virtual pretreatment patient-specific quality assurance of volumetric modulated arc therapy using deep learning

BACKGROUND

Automatic patient-specific quality assurance (PSQA) is recently explored using artificial intelligence approaches, and several studies reported the development of machine learning models for predicting the gamma pass rate (GPR) index only.

PURPOSE

To develop a novel deep learning approach using a generative adversarial network (GAN) to predict the synthetic measured fluence.

METHODS AND MATERIALS

A novel training method called "dual training," which involves the training of the encoder and decoder separately, was proposed and evaluated for cycle GAN (cycle-GAN) and conditional GAN (c-GAN). A total of 164 VMAT treatment plans, including 344 arcs (training data: 262, validation data: 30, and testing data: 52) from various treatment sites, were selected for prediction model development. For each patient, portal-dose-image-prediction fluence from TPS was used as input, and measured fluence from EPID was used as output/response for model training. Predicted GPR was derived by comparing the TPS fluence with the synthetic measured fluence generated by the DL models using gamma evaluation of criteria 2%/2 mm. The performance of dual training was compared against the traditional single-training approach. In addition, we also developed a separate classification model specifically designed to detect automatically three types of errors (rotational, translational, and MU-scale) in the synthetic EPID-measured fluence.

RESULTS

Overall, the dual training improved the prediction accuracy of both cycle-GAN and c-GAN. Predicted GPR results of single training were within 3% for 71.2% and 78.8% of test cases for cycle-GAN and c-GAN, respectively. Moreover, similar results for dual training were 82.7% and 88.5% for cycle-GAN and c-GAN, respectively. The error detection model showed high classification accuracy (>98%) for detecting errors related to rotational and translational errors. However, it struggled to differentiate the fluences with "MU scale error" from "error-free" fluences.

CONCLUSION

We developed a method to automatically generate the synthetic measured fluence and identify errors within them. The proposed dual training improved the PSQA prediction accuracy of both the GAN models, with c-GAN demonstrating superior performance over the cycle-GAN. Our results indicate that the c-GAN with dual training approach combined with error detection model, can accurately generate the synthetic measured fluence for VMAT PSQA and identify the errors. This approach has the potential to pave the way for virtual patient-specific QA of VMAT treatments.

Pretreatment patient-specific quality assurance prediction based on 1D complexity metrics and 3D planning dose: classification, gamma passing rates, and DVH metrics

PURPOSE

Highly modulated radiotherapy plans aim to achieve target conformality and spare organs at risk, but the high complexity of the plan may increase the uncertainty of treatment. Thus, patient-specific quality assurance (PSQA) plays a crucial role in ensuring treatment accuracy and providing clinical guidance. This study aims to propose a prediction model based on complexity metrics and patient planning dose for PSQA results.

MATERIALS AND METHODS

Planning dose, measurement-based reconstructed dose and plan complexity metrics of the 687 radiotherapy plans of patients treated in our institution were collected for model establishing. Global gamma passing rate (GPR, 3%/2mm,10% threshold) of 90% was used as QA criterion. Neural architecture models based on Swin-transformer were adapted to process 3D dose and incorporate 1D metrics to predict QA results. The dataset was divided into training (447), validation (90), and testing (150) sets. Evaluation of predictions was performed using mean absolute error (MAE) for GPR, planning target volume (PTV) HI and PTV CI, mean absolute percentage error (MAPE) for PTV D95, PTV D2 and PTV Dmean, and the area under the receiver operating characteristic (ROC) curve (AUC) for classification. Furthermore, we also compare the prediction results with other models based on either only 1D or 3D inputs.

RESULTS

In this dataset, 72.8% (500/687) plans passed the pretreatment QA under the criterion. On the testing set, our model achieves the highest performance, with the 1D model slightly surpassing the 3D model. The performance results are as follows (combine, 1D, and 3D transformer): The AUCs are 0.92, 0.88 and 0.86 for QA classification. The MAEs of prediction are 0.039, 0.046, and 0.040 for 3D GPR, 0.018, 0.021, and 0.019 for PTV HI, and 0.075, 0.078, and 0.084 for PTV CI. Specifically, for cases with 3D GPRs greater than 90%, the MAE could achieve 0.020 (combine). The MAPE of prediction is 1.23%, 1.52%, and 1.66% for PTV D95, 2.36%, 2.67%, and 2.45% for PTV D2, and 1.46%, 1.70%, and 1.71% for PTV Dmean.

CONCLUSION

The model based on 1D complexity metrics and 3D planning dose could predict pretreatment PSQA results with high accuracy and the complexity metrics play a leading role in the model. Furthermore, dose-volume metric deviations of PTV could be predicted and more clinically valuable information could be provided.

Application of error classification model using indices based on dose distribution for characteristics evaluation of multileaf collimator position errors

This study aims to evaluate the specific characteristics of various multileaf collimator (MLC) position errors that are correlated with the indices using dose distribution. The dose distribution was investigated using the gamma, structural similarity, and dosiomics indices. Cases from the American Association of Physicists in Medicine Task Group 119 were planned, and systematic and random MLC position errors were simulated. The indices were obtained from distribution maps and statistically significant indices were selected. The final model was determined when all values of the area under the curve, accuracy, precision, sensitivity, and specificity were higher than 0.8 (p < 0.05). The dose-volume histogram (DVH) relative percentage difference between the error-free and error datasets was examined to investigate clinical relations. Seven multivariate predictive models were finalized. The common significant dosiomics indices (GLCM Energy and GLRLM_LRHGE) can characterize the MLC position error. In addition, the finalized logistic regression model for MLC position error prediction showed excellent performance with AUC > 0.9. Furthermore, the results of the DVH were related to dosiomics analysis in that it reflects the characteristics of the MLC position error. It was also shown that dosiomics analysis could provide important information on localized dose-distribution differences in addition to DVH information.

Development of a deep learning-based error detection system without error dose maps in the patient-specific quality assurance of volumetric modulated arc therapy

To detect errors in patient-specific quality assurance (QA) for volumetric modulated arc therapy (VMAT), we proposed an error detection method based on dose distribution analysis using unsupervised deep learning approach and analyzed 161 prostate VMAT beams measured with a cylindrical detector. For performing error simulation, in addition to error-free dose distribution, dose distributions containing nine types of error, including multileaf collimator (MLC) positional errors, gantry rotation errors, radiation output errors and phantom setup errors, were generated. Only error-free data were employed for the model training, and error-free and error data were employed for the tests. As a deep learning model, the variational autoencoder (VAE) was adopted. The anomaly of test data was quantified by calculating Mahalanobis distance based on the feature vectors acquired from a trained encoder. Based on this anomaly, test data were classified as 'error-free' or 'any-error.' For comparison with conventional approaches, gamma (γ)-analysis was performed, and supervised learning convolutional neural network (S-CNN) was constructed. Receiver operating characteristic curves were obtained to evaluate their performance with the area under the curve (AUC). For all error types, except systematic MLC positional and radiation output errors, the performance of the methods was in the order of S-CNN ˃ VAE-based ˃ γ-analysis (only S-CNN required error data for model training). For example, in random MLC positional error simulation, the AUC of our method, S-CNN and γ-analysis were 0.699, 0.921 and 0.669, respectively. Our results showed that the VAE-based method has the potential to detect errors in patient-specific VMAT QA.

Image-based features in machine learning to identify delivery errors and predict error magnitude for patient-specific IMRT quality assurance

OBJECTIVE

To identify delivery error type and predict associated error magnitude by image-based features using machine learning (ML).

METHODS

In this study, a total of 40 thoracic plans (including 208 beams) were selected, and four error types with different magnitudes were introduced into the original plans, including 1) collimator misalignment (COLL), 2) monitor unit (MU) variation, 3) systematic multileaf collimator misalignment (MLCS), and 4) random MLC misalignment (MLCR). These dose distributions of portal dose predictions for the original plans were defined as the reference dose distributions (RDD), while those for the error-introduced plans were defined as the error-introduced dose distributions (EDD). Both distributions were calculated for all beams with portal dose image prediction (PDIP). Besides, 14 image-based features were extracted from RDD and EDD of portal dose predictions to obtain the feature vectors. In addition, a random forest was adopted for the multiclass classification task, and regression prediction for error magnitude.

RESULTS

The top five features extracted with the highest weight included 1) the relative displacement in the x direction, 2) the ratio of the absolute minimum residual error to the maximal RDD value, 3) the product of the maximum and minimum residuals, 4) the ratio of the absolute maximum residual error to the maximal RDD value, and 5) the ratio of the absolute mean residual value to the maximal RDD value. The relative displacement in the x direction had the highest weight. The overall accuracy of the five-class classification model was 99.85% for the validation set and 99.30% for the testing set. This model could be applied to the classification of the error-free plan, COLL, MU, MLCS, and MLCR with an accuracy of 100%, 98.4%, 99.9%, 98.0%, and 98.3%, respectively. MLCR had the worst performance in error magnitude prediction (70.1-96.6%), while others had better performance in error magnitude prediction (higher than 93%). In the error magnitude prediction, the mean absolute error (MAE) between predicted error magnitude and actual error ranged from 0.03 to 0.33, with the root mean squared error (RMSE) varying from 0.17 to 0.56 for the validation set. The MAE and RMSE ranged from 0.03 to 0.50 and 0.44 to 0.59 for the test set, respectively.

CONCLUSION

It could be demonstrated in this study that the image-based features extracted from RDD and EDD can be employed to identify different types of delivery errors and accurately predict error magnitude with the assistance of ML techniques. They can be used to associate traditional gamma analysis with clinically based analysis for error classification and magnitude prediction in patient-specific IMRT quality assurance.

A hybrid method to improve efficiency of patient specific SRS and SBRT QA using 3D secondary dose verification

PURPOSE

Patient Specific QA (PSQA) by direct phantom measurement for all intensity modulated radiation therapy (IMRT) cases is labor intensive and an inefficient use of the Medical Physicist's time. The purpose of this work was to develop a hybrid quality assurance (QA) technique utilizing 3D dose verification as a screening tool to determine if a measurement is necessary.

METHODS

This study utilized Sun Nuclear DoseCHECK (DC), a 3D secondary verification software, and Fraction 0, a trajectory log IMRT QA software. Twenty-two Lung stereotactic body radiation therapy (SBRT) and thirty single isocentre multi-lesion SRS (MLSRS) plans were retrospectively analysed in DC. Agreement of DC and the TPS dose for selected dosimetric criteria was recorded. Calculated 95% confidence limits (CL) were used to establish action limits. All cases were delivered and measured using the Sun Nuclear stereotactic radiosurgery (SRS) MapCheck. Trajectory logs of the delivery were used to calculate Fraction 0 results for the same criteria calculated by DC. Correlation of DC and Fraction 0 results were calculated. Phantom measured QA was compared to Fraction 0 QA results for the cases which had DC criteria action limits exceeded.

RESULTS

Correlation of DC and Fraction 0 results were excellent, demonstrating the same action limits could be used for both and DC can predict Fraction 0 results. Based on the calculated action limits, zero lung SBRT cases and six MLSRS cases were identified as requiring a measurement. All plans that passed the DC screening had a passing measurement based PSQA and agreed with Fraction 0 results.

CONCLUSION

Using 95% CL action limits of dosimetric criteria, a 3D secondary dose verification can be used to determine if a measurement is required for PSQA. This method is efficient for it is part of the normal clinical workflow when verifying any clinical treatment. In addition, it can drastically reduce the number of measurements needed for PSQA.

What is the optimal input information for deep learning-based pre-treatment error identification in radiotherapy?

•

The choice of dose comparison method impacts deep learning error identification accuracy most.

•

Simple dose comparison methods are more beneficial than gamma analysis and alternative methods.

•

Mean/standard deviation normalization and high image resolution improve error identification.

Background and purpose

Deep learning (DL) provides high sensitivity for detecting and identifying errors in pre-treatment radiotherapy quality assurance (QA). This work’s objective was to systematically evaluate the impact of different dose comparison and image preprocessing methods on DL model performance for error identification in pre-treatment QA.

Materials and methods

For 53 volumetric modulated arc therapy (VMAT) and 69 stereotactic body radiotherapy (SBRT) treatment plans of lung cancer patients, mechanical errors were simulated (MLC leaf positions, monitor unit scaling, collimator rotation). Two classification levels were assessed: error type (Level 1) and error magnitude (Level 2). Portal dose images with and without errors were compared using standard (gamma analysis), simple (absolute/relative dose difference, ratio) and alternative (distance-to-agreement, structural similarity index, gradient) dose comparison methods. For preprocessing, different normalization methods (min/max and mean/standard deviation) and image resolutions (32 × 32, 64 × 64 and 128 × 128) were evaluated. All possible combinations of classification level, dose comparison, normalization method and image size resulted in 144 input datasets for DL networks for error identification.

Results

Average accuracy was highest for simple dose comparison methods (Level 1: 97.7%, Level 2: 78.1%) while alternative methods scored lowest (Level 1: 91.6%, Level 2: 71.2%). Mean/stdev normalization particularly improved Level 2 classification. Higher image resolution improved error identification, although for SBRT lower image resolution was also sufficient.

Conclusions

The choice of dose comparison method has the largest impact on error identification for pre-treatment QA using DL, compared to image preprocessing. Model performance can improve by using simple dose comparison methods, mean/stdev normalization and high image resolution.

Transit-guided radiation therapy: proof of concept of an on-line technique for correcting position errors using transit portal images

Objective. Transit in vivo dosimetry methods monitor that the dose distribution is delivered as planned. However, they have a limited ability to identify and to quantify the cause of a given disagreement, especially those caused by position errors. This paper describes a proof of concept of a simple in vivo technique to infer a position error from a transit portal image (TPI). Approach. For a given treatment field, the impact of a position error is modeled as a perturbation of the corresponding reference (unperturbed) TPI. The perturbation model determines the patient translation, described by a shift vector, by comparing a given in vivo TPI to the corresponding reference TPI. Patient rotations can also be determined by applying this formalism to independent regions of interest over the patient. Eight treatment plans have been delivered to an anthropomorphic phantom under a large set of couch shifts (<15 mm) and rotations (<10°) to experimentally validate this technique, which we have named Transit-Guided Radiation Therapy (TGRT). Main results. The root mean squared error (RMSE) between the determined and the true shift magnitudes was 1.0/2.4/4.9 mm for true shifts ranging between 0–5/5–10/10–15 mm, respectively. The angular accuracy of the determined shift directions was 12° ± 14°. The RMSE between the determined and the true rotations was 0.5°. The TGRT technique decoupled translations and rotations satisfactorily. In 96% of the cases, the TGRT technique decreased the existing position error. The detection threshold of the TGRT technique was around 1 mm and it was nearly independent of the tumor site, delivery technique, beam energy or patient thickness. Significance. TGRT is a promising technique that not only provides reliable determinations of the position errors without increasing the required equipment, acquisition time or patient dose, but it also adds on-line correction capabilities to existing methods currently using TPIs.

Machine learning models to predict the delivered positions of Elekta multileaf collimator leaves for volumetric modulated arc therapy

PURPOSE

Accurate positioning of multileaf collimator (MLC) leaves during volumetric modulated arc therapy (VMAT) is essential for accurate treatment delivery. We developed a linear regression, support vector machine, random forest, extreme gradient boosting (XGBoost), and an artificial neural network (ANN) for predicting the delivered leaf positions for VMAT plans.

METHODS

For this study, 160 MLC log files from 80 VMAT plans were obtained from a single institution treated on 3 Elekta Versa HD linear accelerators. The gravity vector, X1 and X2 jaw positions, leaf gap, leaf position, leaf velocity, and leaf acceleration were extracted and used as model inputs. The models were trained using 70% of the log files and tested on the remaining 30%. Mean absolute error (MAE), root mean square error (RMSE), the coefficient of determination R² , and fitted line plots showing the relationship between delivered and predicted leaf positions were used to evaluate model performance.

RESULTS

The models achieved the following errors: linear regression (MAE = 0.158 mm, RMSE = 0.225 mm), support vector machine (MAE = 0.141 mm, RMSE = 0.199 mm), random forest (MAE = 0.161 mm, RMSE = 0.229 mm), XGBoost (MAE = 0.185 mm, RMSE = 0.273 mm), and ANN (MAE = 0.361 mm, RMSE = 0.521 mm). A significant correlation between a plan's gamma passing rate (GPR) and the prediction errors of linear regression, support vector machine, and random forest is seen (p < 0.045).

CONCLUSIONS

We examined various models to predict the delivered MLC positions for VMAT plans treated with Elekta linacs. Linear regression, support vector machine, random forest, and XGBoost achieved lower errors than ANN. Models that can accurately predict the individual leaf positions during treatment can help identify leaves that are deviating from the planned position, which can improve a plan's GPR.

Improvement of IMRT QA prediction using imaging-based neural architecture search

PURPOSE

Machine learning has been used to predict the gamma passing rate of Intensity-modulated radiation therapy (IMRT) QA results. In this work, we applied a novel neural architecture search to automatically tune and search for the best deep neural networks instead of using hand-designed deep learning architectures.

METHOD AND MATERIALS

One hundred and eighty-two IMRT plans were created and delivered with portal dosimetry. A total of 1497 fields for multiple treatment sites were delivered and measured by portal imagers. Gamma criteria of 2%/2mm with a 5% threshold were used. Fluence maps calculated for each plan were used as inputs to a convolution neural network (CNN). Auto-Keras was implemented to search for the best CNN architecture for fluence image regression. The network morphism was adopted in the searching process, in which the base models were ResNet and DenseNet. The performance of this CNN approach was compared with tree-based machine learning models previously developed for this application, using the same data set.

RESULTS

The deep-learning-based approach had 98.3% of predictions within 3% of the measured 2%/2mm gamma passing rates with a maximum error of 3.1% and a mean absolute error of less than 1%. Our results show that this novel architecture search approach achieves comparable performance to the machine-learning-based approaches with handcrafted features.

CONCLUSIONS

We implemented a novel CNN model using imaging-based neural architecture for IMRT QA prediction. The imaging-based deep-learning method does not require manual extraction of relevant features and is able to automatically select the best network architecture. This article is protected by copyright. All rights reserved.

Prospective clinical validation of virtual patient-specific quality assurance of VMAT radiation therapy plans

PURPOSE

Performing measurement-based patient-specific quality assurance (PSQA) is recognized as a resource-intensive and time inefficient task in the radiotherapy treatment workflow. Paired with technological refinements in modern radiotherapy, research towards measurement-free PSQA has seen increased interest over the last five years. However, these efforts have not been clinically implemented or prospectively validated in the U.S. We propose a virtual QA (VQA) system and workflow to assess the safety and workload reduction of measurement-free PSQA.

METHODS

An XGBoost machine learning model was designed to predict PSQA outcomes of VMAT plans, represented as percent differences between the measured ion chamber point dose in a phantom and the corresponding planned dose. The final model was deployed within a web application to predict PSQA outcomes of clinical plans within an existing clinical workflow. The application also displays relevant feature importance and plan-specific distribution analyses relative to database plans for documentation and to aid physicist interpretation and evaluation. VQA predictions were prospectively validated over three months of measurements at our clinic to assess safety and efficiency gains.

RESULTS

Over three months, VQA predictions for 445 VMAT plans were prospectively validated at our institution. VQA predictions for these plans had a mean absolute error of 1.08 +/- 0.77%, with a maximum absolute error of 2.98%. Employing a 1% prediction threshold (i.e. plans predicted to have an absolute error of less than 1% would not require a measurement) would yield a 69.2% reduction in QA workload - saving 32.5 hours per month on average - with 81.5%/72.4%/0.81 sensitivity/specificity/AUC at a 3% clinical threshold and 100%/70%/0.93 sensitivity/specificity/AUC at a 4% clinical threshold.

CONCLUSION

This is the first prospective clinical implementation and validation of VQA in the U.S., which we observed to be efficient. Using a conservative threshold, VQA can substantially reduce the number of required measurements for PSQA, leading to more effective allocation of clinical resources.

Deep Neural Network with Structural Similarity Difference and Orientation-based Loss for Position Error Classification in The Radiotherapy of Graves' Ophthalmopathy Patients

Identifying position errors for Graves' ophthalmopathy (GO) patients using electronic portal imaging device (EPID) transmission fluence maps is helpful in monitoring treatment.} However, most of the existing models only extract features from dose difference maps computed from EPID images, which do not fully characterize all information of the positional errors. In addition, the position error has a three-dimensional spatial nature, which has never been explored in previous work. To address the above problems, a deep neural network (DNN) model with structural similarity difference and orientation-based loss is proposed in this paper, which consists of a feature extraction network and a feature enhancement network. To capture more information, three types of Structural SIMilarity (SSIM) sub-index maps are computed to enhance the luminance, contrast, and structural features of EPID images, respectively. These maps and the dose difference maps are fed into different networks to extract radiomic features. To acquire spatial features of the position errors, an orientation-based loss function is proposed for optimal training. It makes the data distribution more consistent with the realistic 3D space by integrating the error deviations of the predicted values in the left-right, superior-inferior, anterior-posterior directions. Experimental results on a constructed dataset demonstrate the effectiveness of the proposed model, compared with other related models and existing state-of-the-art methods.

Analysis of EPID Transmission Fluence Maps Using Machine Learning Models and CNN for Identifying Position Errors in the Treatment of GO Patients

Purpose

To find a suitable method for analyzing electronic portal imaging device (EPID) transmission fluence maps for the identification of position errors in the in vivo dose monitoring of patients with Graves' ophthalmopathy (GO).

Methods

Position errors combining 0-, 2-, and 4-mm errors in the left-right (LR), anterior-posterior (AP), and superior-inferior (SI) directions in the delivery of 40 GO patient radiotherapy plans to a human head phantom were simulated and EPID transmission fluence maps were acquired. Dose difference (DD) and structural similarity (SSIM) maps were calculated to quantify changes in the fluence maps. Three types of machine learning (ML) models that utilize radiomics features of the DD maps (ML 1 models), features of the SSIM maps (ML 2 models), and features of both DD and SSIM maps (ML 3 models) as inputs were used to perform three types of position error classification, namely a binary classification of the isocenter error (type 1), three binary classifications of LR, SI, and AP direction errors (type 2), and an eight-element classification of the combined LR, SI, and AP direction errors (type 3). Convolutional neural network (CNN) was also used to classify position errors using the DD and SSIM maps as input.

Results

The best-performing ML 1 model was XGBoost, which achieved accuracies of 0.889, 0.755, 0.778, 0.833, and 0.532 in the type 1, type 2-LR, type 2-AP, type 2-SI, and type 3 classification, respectively. The best ML 2 model was XGBoost, which achieved accuracies of 0.856, 0.731, 0.736, 0.949, and 0.491, respectively. The best ML 3 model was linear discriminant classifier (LDC), which achieved accuracies of 0.903, 0.792, 0.870, 0.931, and 0.671, respectively. The CNN achieved classification accuracies of 0.925, 0.833, 0.875, 0.949, and 0.689, respectively.

Conclusion

ML models and CNN using combined DD and SSIM maps can analyze EPID transmission fluence maps to identify position errors in the treatment of GO patients. Further studies with large sample sizes are needed to improve the accuracy of CNN.

Radiomics analysis of EPID measurements for patient positioning error detection in thyroid associated ophthalmopathy radiotherapy

PURPOSE

Electronic portal imaging detector (EPID)-based patient positioning verification is an important component of safe radiotherapy treatment delivery. In computer simulation studies, learning-based approaches have proven to be superior to conventional gamma analysis in the detection of positioning errors. To approximate a clinical scenario, the detectability of positioning errors via EPID measurements was assessed using radiomics analysis for patients with thyroid-associated ophthalmopathy.

METHODS

Treatment plans of 40 patients with thyroid-associated ophthalmopathy were delivered to a solid anthropomorphic head phantom. To simulate positioning errors, combinations of 0-, 2-, and 4-mm translation errors in the left-right (LR), superior-inferior (SI), and anterior-posterior (AP) directions were introduced to the phantom. The positioning errors-induced dose differences between measured portal dose images were used to predict the magnitude and direction of positioning errors. The detectability of positioning errors was assessed via radiomics analysis of the dose differences. Three classification models-support vector machine (SVM), k-nearest neighbors (KNN), and XGBoost-were used for the detection of positioning errors (positioning errors larger or smaller than 3 mm in an arbitrary direction) and direction classification (positioning errors larger or smaller than 3 mm in a specific direction). The receiver operating characteristic curve and the area under the ROC curve (AUC) were used to evaluate the performance of classification models.

RESULTS

For the detection of positioning errors, the AUC values of SVM, KNN, and XGBoost models were all above 0.90. For LR, SI, and AP direction classification, the highest AUC values were 0.76, 0.91, and 0.80, respectively.

CONCLUSIONS

Combined radiomics and machine learning approaches are capable of detecting the magnitude and direction of positioning errors from EPID measurements. This study is a further step toward machine learning-based positioning error detection during treatment delivery with EPID measurements.

Molecular subtyping of diffuse gliomas using magnetic resonance imaging: comparison and correlation between radiomics and deep learning

OBJECTIVES

The molecular subtyping of diffuse gliomas is important. The aim of this study was to establish predictive models based on preoperative multiparametric MRI.

METHODS

A total of 1016 diffuse glioma patients were retrospectively collected from Beijing Tiantan Hospital. Patients were randomly divided into the training (n = 780) and validation (n = 236) sets. According to the 2016 WHO classification, diffuse gliomas can be classified into four binary classification tasks (tasks I-IV). Predictive models based on radiomics and deep convolutional neural network (DCNN) were developed respectively, and their performances were compared with receiver operating characteristic (ROC) curves. Additionally, the radiomics and DCNN features were visualized and compared with the t-distributed stochastic neighbor embedding technique and Spearman's correlation test.

RESULTS

In the training set, areas under the curves (AUCs) of the DCNN models (ranging from 0.99 to 1.00) outperformed the radiomics models in all tasks, and the accuracies of the DCNN models (ranging from 0.90 to 0.94) outperformed the radiomics models in tasks I, II, and III. In the independent validation set, the accuracies of the DCNN models outperformed the radiomics models in all tasks (0.74-0.83), and the AUCs of the DCNN models (0.85-0.89) outperformed the radiomics models in tasks I, II, and III. DCNN features demonstrated more superior discriminative capability than the radiomics features in feature visualization analysis, and their general correlations were weak.

CONCLUSIONS

Both the radiomics and DCNN models could preoperatively predict the molecular subtypes of diffuse gliomas, and the latter performed better in most circumstances.

KEY POINTS

• The molecular subtypes of diffuse gliomas could be predicted with MRI. • Deep learning features tend to outperform radiomics features in large cohorts. • The correlation between the radiomics features and DCNN features was low.

Error detection model developed using a multi-task convolutional neural network in patient-specific quality assurance for volumetric-modulated arc therapy

PURPOSE

In patient-specific quality assurance (QA) for static beam intensity-modulated radiation therapy (IMRT), machine-learning-based dose analysis methods have been developed to identify the cause of an error as an alternative to gamma analysis. Although these new methods have revealed that the cause of the error can be identified by analyzing the dose distribution obtained from the two-dimensional detector, they have not been extended to the analysis of volumetric-modulated arc therapy (VMAT) QA. In this study, we propose a deep learning approach to detect various types of errors in patient-specific VMAT QA.

METHODS

A total of 161 beams from 104 prostate VMAT plans were analyzed. All beams were measured using a cylindrical detector (Delta4; ScandiDos, Uppsala, Sweden), and predicted dose distributions in a cylindrical phantom were calculated using a treatment planning system (TPS). In addition to the error-free plan, we simulated 12 types of errors: two types of multileaf collimator positional errors (systematic or random leaf offset of 2 mm), two types of monitor unit (MU) scaling errors (±3%), two types of gantry rotation errors (±2° in clockwise and counterclockwise direction), and six types of phantom setup errors (±1 mm in lateral, longitudinal, and vertical directions). The error-introduced predicted dose distributions were created by editing the calculated dose distributions using a TPS with in-house software. Those 13 types of dose difference maps, consisting of an error-free map and 12 error maps, were created from the measured and predicted dose distributions and were used to train the convolutional neural network (CNN) model. Our model was a multi-task model that individually detected each of the 12 types of errors. Two datasets, Test sets 1 and 2, were prepared to evaluate the performance of the model. Test set 1 consisted of 13 types of dose maps used for training, whereas Test set 2 included the dose maps with 25 types of errors in addition to the error-free dose map. The dose map, which introduced 25 types of errors, was generated by combining two of the 12 types of simulated errors. For comparison with the performance of our model, gamma analysis was performed for Test sets 1 and 2 with the criteria set to 3%/2 mm and 2%/1 mm (dose difference/distance to agreement).

RESULTS

For Test set 1, the overall accuracy of our CNN model, gamma analysis with the criteria set to 3%/2 mm, and gamma analysis with the criteria set to 2%/1 mm was 0.92, 0.19, and 0.81, respectively. Similarly, for Test set 2, the overall accuracy was 0.44, 0.42, and 0.95, respectively. Our model outperformed gamma analysis in the classification of dose maps containing a single type error, and the performance of our model was inferior in the classification of dose maps containing compound errors.

CONCLUSIONS

A multi-task CNN model for detecting errors in patient-specific VMAT QA using a cylindrical measuring device was constructed, and its performance was evaluated. Our results demonstrate that our model was effective in identifying the error type in the dose map for VMAT QA.

Prediction of an oxygen extraction fraction map by convolutional neural network: validation of input data among MR and PET images

Purpose

Oxygen extraction fraction (OEF) is a biomarker for the viability of brain tissue in ischemic stroke. However, acquisition of the OEF map using positron emission tomography (PET) with oxygen-15 gas is uncomfortable for patients because of the long fixation time, invasive arterial sampling, and radiation exposure. We aimed to predict the OEF map from magnetic resonance (MR) and PET images using a deep convolutional neural network (CNN) and to demonstrate which PET and MR images are optimal as inputs for the prediction of OEF maps.

Methods

Cerebral blood flow at rest (CBF) and during stress (sCBF), cerebral blood volume (CBV) maps acquired from oxygen-15 PET, and routine MR images (T1-, T2-, and T2*-weighted images) for 113 patients with steno-occlusive disease were learned with U-Net. MR and PET images acquired from the other 25 patients were used as test data. We compared the predicted OEF maps and intraclass correlation (ICC) with the real OEF values among combinations of MRI, CBF, CBV, and sCBF.

Results

Among the combinations of input images, OEF maps predicted by the model learned with MRI, CBF, CBV, and sCBF maps were the most similar to the real OEF maps (ICC: 0.597 ± 0.082). However, the contrast of predicted OEF maps was lower than that of real OEF maps.

Conclusion

These results suggest that the deep CNN learned useful features from CBF, sCBF, CBV, and MR images and predict qualitatively realistic OEF maps. These findings suggest that the deep CNN model can shorten the fixation time for ¹⁵O PET by skipping ¹⁵O₂ scans. Further training with a larger data set is required to predict accurate OEF maps quantitatively.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11548-021-02356-7.

Use of Receiver Operating Curve Analysis and Machine Learning With an Independent Dose Calculation System Reduces the Number of Physical Dose Measurements Required for Patient-Specific Quality Assurance

PURPOSE

Our purpose was to assess the use of machine learning methods and Mobius 3D (M3D) dose calculation software to reduce the number of physical ion chamber (IC) dose measurements required for patient-specific quality assurance during corona virus disease 2019.

METHODS AND MATERIALS

In this study, 1464 inversely planned treatments using Pinnacle or Raystation treatment planning software (TPS) were delivered using Elekta Versa HD and Varian Truebeam and Truebeam STx linear accelerators between June 2018 and November 2019. For each plan, an independent dose calculation was performed using M3D, and an absolute dose measurement was taken using a Pinpoint IC inside the Mobius phantom. The point dose differences between the TPS and M3D calculation and between TPS and IC measurements were calculated. Agreement between the TPS and IC was used to define the ground truth plan failure. To reduce the on-site personnel during the pandemic, 2 methods of receiver operating characteristic analysis (n = 1464) and machine learning (n = 603) were used to identify patient plans that would require physical dose measurements.

RESULTS

In the receiver operating characteristic analysis, a predelivery M3D difference threshold of 3% identified plans that failed an IC measurement at a 4% threshold with 100% sensitivity and 76.3% specificity. This indicates that fewer than 25% of plans required a physical dose measurement. A threshold of 1% on a machine learning model was able to identify plans that failed an IC measurement at a 3% threshold with 100% sensitivity and 54.3% specificity, leading to fewer than 50% of plans that required a physical dose measurement.

CONCLUSIONS

It is possible to identify plans that are more likely to fail IC patient-specific quality assurance measurements before delivery. This possibly allows for a reduction of physical measurements taken, freeing up significant clinical resources and reducing the required amount of on-site personnel while maintaining patient safety.

The structural similarity index for IMRT quality assurance: radiomics-based error classification

PURPOSE

The implementation of radiomics and machine learning (ML) techniques on analyzing two-dimensional gamma maps has been demonstrated superior to the conventional gamma analysis for error identification in intensity modulated radiotherapy (IMRT) quality assurance (QA). Recently, the Structural SIMilarity (SSIM) sub-index maps were shown to be able to reveal the error types of the dose distributions. In this study, we aimed to apply radiomics analysis on SSIM sub-index maps and develop ML models to classify delivery errors in patient-specific dynamic IMRT QA.

METHODS

Twenty-one sliding-window IMRT plans of 180 beams for three treatment sites were involved in this study. Four types of machine-related errors of various magnitudes were simulated for each beam at each control point, including the monitor unit (MU) variations, same-directional and opposite-directional shifts of the multileaf collimators (MLCs) and random mispositioning of the MLCs. In the QA process, a total of 1620 portal dose (PD) images were acquired for the beams with and without errors. The predicted PD images of the original beams were set as references. To quantify the agreement between a measured PD image and the corresponding predicted PD image, four difference maps including three SSIM sub-index maps, and one dose difference-derived map were calculated. Then, radiomic features were extracted from the four difference maps of each measured PD image. We tested four typical classifiers including linear discriminant classifier (LDC), two supporting vector machine (SVM) classifiers, and random forest (RF) for this multiclass classification task. A nested cross-validation scheme was used for model evaluations, where the SVM recursive feature elimination method was applied for feature selection. Finally, the performance of the ML model on identifying the error-free and the erroneous cases was compared to that of the conventional gamma analysis.

RESULTS

The statistics of the selected features showed that all of the difference maps and the feature categories made balanced contributions to solve this classification task. Best performance was achieved by the Linear-SVM model with average overall classification accuracy of 0.86. Specifically, the average classification accuracies of the shift, opening, and the random errors were around 0.9. Moreover, ~80% of error-free and MU errors were correctly classified. Using gamma analysis, the 3 mm/3% criterion was found insensitive to errors (sensitivity was only 0.33). Although the sensitivity to errors with the 2 mm/2% criterion increased to 0.79, still 8% worse than that of the ML model.

CONCLUSIONS

We proposed an ML-based method for machine-related error identification in patient-specific dynamic IMRT QA, where radiomic analysis on SSIM sub-index maps were used for feature extraction. With extensive validation to select the best features and classifiers, high accuracies in error classification were achieved. Compared with the conventional gamma threshold method, this approach has great potential in error identification for the patient-specific IMRT QA process.