Radiation Therapy Deficiencies Identified During On-Site Dosimetry Visits by the Imaging and Radiation Oncology Core Houston Quality Assurance Center

Which failures do patient-specific quality assurance systems need to catch?

BACKGROUND

The Joint AAPM-ESTRO TG-360 is developing a quantitative framework to evaluate treatment verification systems used for patient-specific quality assurance (PSQA). A subgroup was commissioned to determine which potential failure modes had the greatest risk to treatment quality and safety, and therefore should be evaluated as part of the PSQA verification.

PURPOSE

To create an extensive database of potential radiotherapy failure modes that should be detected by PSQA and to determine their relative importance for maximizing treatment quality.

METHODS

The subgroup consisted of eight physicists from seven countries, including representatives from three international quality assurance groups. We collected error reports from RO-ILS, SAFRON, AAPM TG publications, and other literature, including international audits. We focused on the subset of failure modes that impact whether the planned dose matches the dose received by the patient. We performed a failure-mode-and-effects analysis (FMEA), estimating the severity (S), occurrence (O), and detectability (D) of each failure mode. Detectability was scored assuming that PSQA was not done but other routine clinical QA was performed, which allowed us to see the importance of PSQA for detecting each specific failure mode. We analyzed the risk priority number (RPN = O*S*D), O*S, and severity rankings to determine the priority of each failure mode.

RESULTS

We collected 394 error reports, which we categorized into 33 failure modes that underwent FMEA. Five failure modes were in the top ranks for both RPN and O*S analysis: four involving treatment planning system (TPS) commissioning and one regarding patient model errors. The highest-ranking RPN failure modes were: TPS algorithm limitations, TPS commissioning errors [multileaf collimator (MLC) modeling, output factor, percent-depth-dose/tissue-maximum-ratio (PDD/TMR), off-axis factor], and patient weight variation. The highest O*S failure modes were similar, with the addition of external patient position variation and incorrect linear accelerator isocenter and cGy/monitor units calibration. RPN and O*S analyses prioritized failure modes that impacted multiple patients with high occurrence and detectability scores, while severity analysis gave higher priority to single-patient modes with high severity scores. The highest-ranking severity modes were MLC sequence deletion, collision, and TPS isocenter incorrect.

CONCLUSION

We have developed a list of failure modes critical to be detected during PSQA and ranked them in order of importance. The top failure modes emphasize the importance of utilizing a variety of treatment verification systems for PSQA, from secondary dose calculation through in-vivo dosimetry, in order to detect all possible errors. For failure modes in the top quartile, PSQA is critical. Without adequate PSQA, these errors may go undetected unless caught by an external audit. This analysis can be useful for optimizing PSQA workflows and for designing evaluations of treatment verification systems, and will be used by the Joint AAPM-ESTRO TG-360 to determine an appropriate validation strategy.

Is the Imaging Radiation Oncology Core Head and Neck Credentialing Phantom an Effective Surrogate for Different Anatomic Sites?

PURPOSE

The Imaging Radiation Oncology Core (IROC) head and neck (H&N) phantom is used to credential institutions for intensity modulated radiation therapy delivery for all anatomic sites where delivery of modulated therapy is a primary challenge. This study evaluated how appropriate the use of this phantom is for varied clinical anatomy by evaluating how closely the IROC H&N phantom described clinical dose errors from beam modeling compared with various anatomic sites.

METHODS AND MATERIALS

The multileaf collimator (MLC) offset, transmission, percent depth dose, and 7 additional beam modeling parameters for a Varian accelerator were modified in RayStation to match community data at the 2.5th, 25th, 50th, 75th, and 97.5th percentile levels. Modifications were evaluated on 25 H&N phantom cases and 25 clinical cases (H&N, prostate, lung, mesothelioma, and brain), generating 2000 plan perturbations. Differences in mean dose delivered to clinical target volumes and maximum dose to organs at risk were compared between phantom and clinical plans to assess the relationship between dose deviations in phantom versus clinical target volumes and as a function of 18 different complexity metrics.

RESULTS

Perturbations to MLC offset and transmission parameters demonstrated the greatest impact on dose accuracy for phantom and clinical plans (for all anatomic sites). The phantom demonstrated equivalent or greater sensitivity to these parameter perturbations compared with clinical sites, largely aligning with treatment complexity. The mean MLC gap best described the impact of errors in treatment planning system beam modeling parameters in phantom plans and clinical plans from various anatomic sites.

CONCLUSIONS

When compared across various anatomic sites, the IROC H&N credentialing phantom exhibited similar or greater sensitivity to errors in the treatment planning system. As such, it is a suitable surrogate device for assessing institutional performance across various anatomic sites. If an institution successfully irradiates the phantom, that result confers confidence that intensity modulated radiation therapy to a wide range of anatomic sites can be successfully delivered by the institution.

Dosimetric impact of calibration coefficients determined using linear accelerator photon and electron beams for ionization chamber in an on-site dosimetry audit

This study aimed to clarify the dosimetric impact of calibration beam quality for calibration coefficients of the absorbed dose to water for an ionization chamber in an on-site dosimetry audit. Institution-measured doses of 200 photon and 184 electron beams were compared with the measured dose using one year data before and after the calibration of the ionization chamber used. For photon and electron reference dosimetry, the agreements of the institution-measured dose against two measured doses in this audit were evaluated using the calibration coefficients determined using 60Co (${N}_{D,\mathrm{w},{}^{60}\mathrm{Co}}$) and linear accelerator (linac) (${N}_{D,\mathrm{w},Q}$) beams. For electron reference dosimetry, the agreement of two institution-measured doses against the measured dose was evaluated using${N}_{D,\mathrm{w},Q}$. Institution-measured doses were evaluated using direct- and cross-calibration coefficients. For photon reference dosimetry, the mean differences and standard deviation (SD) of institution-measured dose against the measured dose using ${N}_{D,\mathrm{w},{}^{60}\mathrm{Co}}$ and ${N}_{D,\mathrm{w},Q}$ were -0.1% ± 0.4% and -0.3% ± 0.4%, respectively. For electron reference dosimetry, the mean differences and SD of institution-measured dose using the direct-calibration coefficient against the measured dose using ${N}_{D,\mathrm{w},{}^{60}\mathrm{Co}}$ and ${N}_{D,\mathrm{w},Q}$ were 1.3% ± 0.8% and 0.8% ± 0.8%, respectively. Further, the mean differences and SD of institution-measured dose using the cross-calibration coefficient against the measured dose using ${N}_{D,\mathrm{w},Q}$ were -0.1% ± 0.6%. For photon beams, the dosimetric impact of introducing calibration coefficients determined using linac beams was small. For electron beams, it was larger, and the measured dose using ${N}_{D,\mathrm{w},Q}$ was most consistent with the institution-measured dose, which was evaluated using a cross-calibration coefficient.

AAPM WGTG51 Report 385: Addendum to the AAPM's TG-51 protocol for clinical reference dosimetry of high-energy electron beams

An Addendum to the AAPM's TG-51 protocol for the determination of absorbed dose to water is presented for electron beams with energies between 4 MeV and 22 MeV (1.70 cm ≤ R 50 ≤ 8.70 cm $1.70\nobreakspace {\rm cm} \le R_{\text{50}} \le 8.70\nobreakspace {\rm cm}$ ). This updated formalism allows simplified calibration procedures, including the use of calibrated cylindrical ionization chambers in all electron beams without the use of a gradient correction. Newk Q $k_{Q}$ data are provided for electron beams based on Monte Carlo simulations. Implementation guidance is provided. Components of the uncertainty budget in determining absorbed dose to water at the reference depth are discussed. Specifications for a reference-class chamber in electron beams include chamber stability, settling, ion recombination behavior, and polarity dependence. Progress in electron beam reference dosimetry is reviewed. Although this report introduces some major changes (e.g., gradient corrections are implicitly included in the electron beam quality conversion factors), they serve to simplify the calibration procedure. Results for absorbed dose per linac monitor unit are expected to be up to approximately 2 % higher using this Addendum compared to using the original TG-51 protocol.

Universal evaluation of MLC models in treatment planning systems based on a common set of dynamic tests

PURPOSE

To demonstrate the feasibility of characterising MLCs and MLC models implemented in TPSs using a common set of dynamic beams.

MATERIALS AND METHODS

A set of tests containing synchronous (SG) and asynchronous sweeping gaps (aSG) was distributed among twenty-five participating centres. Doses were measured with a Farmer-type ion chamber and computed in TPSs, which provided a dosimetric characterisation of the leaf tip, tongue-and-groove, and MLC transmission of each MLC, as well as an assessment of the MLC model in each TPS. Five MLC types and four TPSs were evaluated, covering the most frequent combinations used in radiotherapy departments.

RESULTS

Measured differences within each MLC type were minimal, while large differences were found between MLC models implemented in clinical TPSs. This resulted in some concerning discrepancies, especially for the HD120 and Agility MLCs, for which differences between measured and calculated doses for some MLC-TPS combinations exceeded 10%. These large differences were particularly evident for small gap sizes (5 and 10 mm), as well as for larger gaps in the presence of tongue-and-groove effects. A much better agreement was found for the Millennium120 and Halcyon MLCs, differences being within ± 5% and ± 2.5%, respectively.

CONCLUSIONS

The feasibility of using a common set of tests to assess MLC models in TPSs was demonstrated. Measurements within MLC types were very similar, but TPS dose calculations showed large variations. Standardisation of the MLC configuration in TPSs is necessary. The proposed procedure can be readily applied in radiotherapy departments and can be a valuable tool in IMRT and credentialing audits.

Integrating plan complexity and dosiomics features with deep learning in patient-specific quality assurance for volumetric modulated arc therapy

PURPOSE

To investigate the feasibility and performance of deep learning (DL) models combined with plan complexity (PC) and dosiomics features in the patient-specific quality assurance (PSQA) for patients underwent volumetric modulated arc therapy (VMAT).

METHODS

Total of 201 VMAT plans with measured PSQA results were retrospectively enrolled and divided into training and testing sets randomly at 7:3. PC metrics were calculated using house-built algorithm based on Matlab. Dosiomics features were extracted and selected using Random Forest (RF) from planning target volume (PTV) and overlap regions with 3D dose distributions. The top 50 dosiomics and 5 PC features were selected based on feature importance screening. A DL DenseNet was adapted and trained for the PSQA prediction.

RESULTS

The measured average gamma passing rate (GPR) of these VMAT plans was 97.94% ± 1.87%, 94.33% ± 3.22%, and 87.27% ± 4.81% at the criteria of 3%/3 mm, 3%/2 mm, and 2%/2 mm, respectively. Models with PC features alone demonstrated the lowest area under curve (AUC). The AUC and sensitivity of PC and dosiomics (D) combined model at 2%/2 mm were 0.915 and 0.833, respectively. The AUCs of DL models were improved from 0.943, 0.849, 0.841 to 0.948, 0.890, 0.942 in the combined models (PC + D + DL) at 3%/3 mm, 3%/2 mm and 2%/2 mm, respectively. A best AUC of 0.942 with a sensitivity, specificity and accuracy of 100%, 81.8%, and 83.6% was achieved with combined model (PC + D + DL) at 2%/2 mm.

CONCLUSIONS

Integrating DL with dosiomics and PC metrics is promising in the prediction of GPRs in PSQA for patients underwent VMAT.

Quality Assurance in Clinical Trials Requiring Radiation Therapy in Sub-Saharan Africa

PURPOSE

Given the increasing availability of radiation therapy in sub-Saharan Africa, clinical trials that include radiation therapy are likely to grow. Ensuring appropriate delivery of radiation therapy through rigorous quality assurance is an important component of clinical trial execution. We reviewed the process for credentialing radiation therapy sites and radiation therapy quality assurance through the Imaging and Radiation Oncology Core (IROC) Houston Quality Assurance Center for AIDS Malignancy Consortium (AMC)-081, a multicenter study of cisplatin and radiation therapy for women with locally advanced cervical cancer living with HIV, conducted by the AIDS Malignancy Consortium at 2 sites in South Africa and Zimbabwe.

METHODS AND MATERIALS

Women living with HIV with newly diagnosed stage IB2, IIA (>4 cm), IIB-IVA cervical carcinoma (per the 2009 International Federation of Gynecology and Obstetrics [FIGO] staging classifications) were enrolled in AMC-081. They received 3-dimensional conformal external beam radiation therapy (EBRT) to the pelvis (41.4-45 Gy) using a linear accelerator, high-dose-rate brachytherapy (6-9 Gy to point A with each fraction and up to 4 fractions), and concurrent weekly cisplatin (40 mg/m2). IROC reviewed EBRT and brachytherapy quality assurance records after treatment.

RESULTS

All of the 38 women enrolled in AMC-081 received ±5% of the protocol-specified prescribed dose of EBRT. Geometry of brachytherapy applicator placement was scored as per protocol in all implants. Doses to points A and B, International Commission on Radiation Units and Measurements (ICRU) bladder, or ICRU rectum required correction by IROC in >50% of the implants. In the final evaluation, 58% of participants (n = 22) were treated per protocol, 40% (n = 15) had minor protocol deviations, and 3% (n = 1) had major protocol deviations. No records were received within 60 days of treatment completion as requested in the protocol.

CONCLUSIONS

Major radiation therapy deviations were low, but timely submission of radiation therapy data did not occur. Future studies, especially those that include specialized radiation therapy techniques such as stereotactic or intensity-modulated radiation therapy, will require pathways to ensure timely and adequate quality assurance.

Quality improvements in radiation oncology clinical trials

Clinical trials have become the primary mechanism to validate process improvements in oncology clinical practice. Over the past two decades there have been considerable process improvements in the practice of radiation oncology within the structure of a modern department using advanced technology for patient care. Treatment planning is accomplished with volume definition including fusion of multiple series of diagnostic images into volumetric planning studies to optimize the definition of tumor and define the relationship of tumor to normal tissue. Daily treatment is validated by multiple tools of image guidance. Computer planning has been optimized and supported by the increasing use of artificial intelligence in treatment planning. Informatics technology has improved, and departments have become geographically transparent integrated through informatics bridges creating an economy of scale for the planning and execution of advanced technology radiation therapy. This serves to provide consistency in department habits and improve quality of patient care. Improvements in normal tissue sparing have further improved tolerance of treatment and allowed radiation oncologists to increase both daily and total dose to target. Radiation oncologists need to define a priori dose volume constraints to normal tissue as well as define how image guidance will be applied to each radiation treatment. These process improvements have enhanced the utility of radiation therapy in patient care and have made radiation therapy an attractive option for care in multiple primary disease settings. In this chapter we review how these changes have been applied to clinical practice and incorporated into clinical trials. We will discuss how the changes in clinical practice have improved the quality of clinical trials in radiation therapy. We will also identify what gaps remain and need to be addressed to offer further improvements in radiation oncology clinical trials and patient care.

Prioritizing clinical trial quality assurance for photons and protons: A failure modes and effects analysis (FMEA) comparison

BACKGROUND AND PURPOSE

The Global Clinical Trials RTQA Harmonization Group (GHG) set out to evaluate and prioritize clinical trial quality assurance.

METHODS

The GHG compiled a list of radiotherapy quality assurance (QA) tests performed for proton and photon therapy clinical trials. These tests were compared between modalities to assess whether there was a need for different types of assessments per modality. A failure modes and effects analysis (FMEA) was performed to assess the risk of each QA failure.

RESULTS

The risk analysis showed that proton and photon therapy shared four out of five of their highest-risk failures (end-to-end anthropomorphic phantom test, phantom tests using respiratory motion, pre-treatment patient plan review of contouring/outlining, and on-treatment/post-treatment patient plan review of dosimetric coverage). While similar trends were observed, proton therapy had higher risk failures, driven by higher severity scores. A sub-analysis of occurrence × severity scores identified high-risk scores to prioritize for improvements in RTQA detectability. A novel severity scaler was introduced to account for the number of patients affected by each failure. This scaler did not substantially alter the ranking of tests, but it elevated the QA program evaluation to the top 20th percentile. This is the first FMEA performed for clinical trial quality assurance.

CONCLUSION

The identification of high-risk errors associated with clinical trials is valuable to prioritize and reduce errors in radiotherapy and improve the quality of trial data and outcomes, and can be applied to optimize clinical radiotherapy QA.

AAPM MEDICAL PHYSICS PRACTICE GUIDELINE 5.b: Commissioning and QA of treatment planning dose calculations-Megavoltage photon and electron beams

The American Association of Physicists in Medicine (AAPM) is a nonprofit professional society whose primary purposes are to advance the science, education, and professional practice of medical physics. The AAPM has more than 8000 members and is the principal organization of medical physicists in the United States. The AAPM will periodically define new practice guidelines for medical physics practice to help advance the science of medical physics and to improve the quality of service to patients throughout the United States. Existing medical physics practice guidelines will be reviewed for the purpose of revision or renewal, as appropriate, on their fifth anniversary or sooner. Each medical physics practice guideline represents a policy statement by the AAPM, has undergone a thorough consensus process in which it has been subjected to extensive review, and requires the approval of the Professional Council. The medical physics practice guidelines recognize that the safe and effective use of diagnostic and therapeutic radiology requires specific training, skills, and techniques, as described in each document. Reproduction or modification of the published practice guidelines and technical standards by those entities not providing these services is not authorized. The following terms are used in the AAPM practice guidelines:

Must and Must Not: Used to indicate that adherence to the recommendation is considered necessary to conform to this practice guideline. While must is the term to be used in the guidelines, if an entity that adopts the guideline has shall as the preferred term, the AAPM considers that must and shall have the same meaning.

Should and Should Not: Used to indicate a prudent practice to which exceptions may occasionally be made in appropriate circumstances.

Application programming interface guided QA plan generation and analysis automation

Purpose

Linear accelerator quality assurance (QA) in radiation therapy is a time consuming but fundamental part of ensuring the performance characteristics of radiation delivering machines. The goal of this work is to develop an automated and standardized QA plan generation and analysis system in the Oncology Information System (OIS) to streamline the QA process.

Methods

Automating the QA process includes two software components: the AutoQA Builder to generate daily, monthly, quarterly, and miscellaneous periodic linear accelerator QA plans within the Treatment Planning System (TPS) and the AutoQA Analysis to analyze images collected on the Electronic Portal Imaging Device (EPID) allowing for a rapid analysis of the acquired QA images. To verify the results of the automated QA analysis, results were compared to the current standard for QA assessment for the jaw junction, light‐radiation coincidence, picket fence, and volumetric modulated arc therapy (VMAT) QA plans across three linacs and over a 6‐month period.

Results

The AutoQA Builder application has been utilized clinically 322 times to create QA patients, construct phantom images, and deploy common periodic QA tests across multiple institutions, linear accelerators, and physicists. Comparing the AutoQA Analysis results with our current institutional QA standard the mean difference of the ratio of intensity values within the field‐matched junction and ball‐bearing position detection was 0.012 ± 0.053 (P = 0.159) and is 0.011 ± 0.224 mm (P = 0.355), respectively. Analysis of VMAT QA plans resulted in a maximum percentage difference of 0.3%.

Conclusion

The automated creation and analysis of quality assurance plans using multiple APIs can be of immediate benefit to linear accelerator quality assurance efficiency and standardization. QA plan creation can be done without following tedious procedures through API assistance, and analysis can be performed inside of the clinical OIS in an automated fashion.

Simplified sigmoidal curve fitting for a 6 MV FFF photon beam of the Halcyon to determine the field size for beam commissioning and quality assurance

Background

An O-ring gantry-type linear accelerator (LINAC) with a 6-MV flattening filter-free (FFF) photon beam, Halcyon, includes a reference beam that contains representative information such as the percent depth dose, profile and output factor for commissioning and quality assurance. However, because it does not provide information about the field size, we proposed a method to determine all field sizes according to all depths for radiation therapy using simplified sigmoidal curve fitting (SCF).

Methods

After mathematical definition of the SCF using four coefficients, the defined curves were fitted to both the reference data (RD) and the measured data (MD). For good agreement between the fitting curve and the profiles in each data set, the field sizes were determined by identifying the maximum point along the third derivative of the fitting curve. The curve fitting included the field sizes for beam profiles of 2 × 2, 4 × 4, 6 × 6, 8 × 8, 10 × 10, 20 × 20 and 28 × 28 cm² as a function of depth (at 1.3, 5, 10 and 20 cm). The field size results from the RD were compared with the results from the MD using the same condition.

Results

All fitting curves show goodness of fit, R², values that are greater than 0.99. The differences in field size between the RD and the MD were within the range of 0 to 0.2 cm. The smallest difference in the field sizes at a depth of 10 cm, which is a surface-to-axis distance, was reported.

Conclusion

Application of the SCF method has been proven to accurately capture the field size of the preconfigured RD and the measured FFF photon beam data for the Halcyon system. The current work can be useful for beam commissioning as a countercheck methodology to determine the field size from RD in the treatment planning system of a newly installed Halcyon system and for routine quality assurance to ascertain the correctness of field sizes for clinical use of the Halcyon system.

Error detection and classification in patient-specific IMRT QA with dual neural networks

PURPOSE

Despite being the standard metric in patient-specific quality assurance (QA) for intensity-modulated radiotherapy (IMRT), gamma analysis has two shortcomings: (a) it lacks sensitivity to small but clinically relevant errors (b) it does not provide efficient means to classify the error sources. The purpose of this work is to propose a dual neural network method to achieve simultaneous error detection and classification in patient-specific IMRT QA.

METHODS

For a pair of dose distributions, we extracted the dose difference histogram (DDH) for the low dose gradient region and two signed distance-to-agreement (sDTA) maps (one in x direction and one in y direction) for the high dose gradient region. An artificial neural network (ANN) and a convolutional neural network (CNN) were designed to analyze the DDH and the two sDTA maps, respectively. The ANN was trained to detect and classify six classes of dosimetric errors: incorrect multileaf collimator (MLC) transmission (±1%) and four types of monitor unit (MU) scaling errors (±1% and ±2%). The CNN was trained to detect and classify seven classes of spatial errors: incorrect effective source size, 1 mm MLC leaf bank overtravel or undertravel, 2 mm single MLC leaf overtravel or undertravel, and device misalignment errors (1 mm in x- or y direction). An in-house planar dose calculation software was used to simulate measurements with errors and noise introduced. Both networks were trained and validated with 13 IMRT plans (totaling 88 fields). A fivefold cross-validation technique was used to evaluate their accuracy.

RESULTS

Distinct features were found in the DDH and the sDTA maps. The ANN perfectly identified all four types of MU scaling errors and the specific accuracies for the classes of no error, MLC transmission increase, MLC transmission decrease were 98.9%, 96.6%, and 94.3%, respectively. For the CNN, the largest confusion occurred between the 1-mm-MLC bank overtravel class and the 1-mm-device alignment error in x-direction class, which brought the specific accuracies down to 90.9% and 92.0%, respectively. The specific accuracy for the 2-mm-single MLC leaf undertravel class was 93.2% as it misclassified 5.7% of the class as being error free (false negative). Otherwise, the specific accuracy was above 95%. The overall accuracies across the fivefold were 98.3 ± 0.7% and 95.6% ± 1.5% for the ANN and the CNN, respectively.

CONCLUSIONS

Both the DDH and the sDTA maps are suitable features for error classification in IMRT QA. The proposed dual neural network method achieved simultaneous error detection and classification with excellent accuracy. It could be used in complement with the gamma analysis to potentially shift the IMRT QA paradigm from passive pass/fail analysis to active error detection and root cause identification.

Dose calculation errors as a component of failing IROC lung and spine phantom irradiations

PURPOSE

Between July 2013 and August 2019, 22% of the imaging and radiation oncology core (IROC) spine, and 15% of the moving lung phantom irradiations have failed to meet established acceptability criteria. The spine phantom simulates a highly modulated stereotactic body radiation therapy (SBRT) case, whereas the lung phantom represents a low-to-none modulation moving target case. In this study, we assessed the contribution of dose calculation errors to these phantom results and evaluated their effects on failure rates.

METHODS

We evaluated dose calculation errors by comparing the calculation accuracy of various institutions' treatment planning systems (TPSs) vs IROC-Houston's previously established independent dose recalculation system (DRS). Each calculation was compared with the measured dose actually delivered to the phantom; cases in which the recalculation was more accurate were interpreted as a deficiency in the institution's TPS. A total of 258 phantom irradiation plans (172 lung and 86 spine) were recomputed.

RESULTS

Overall, the DRS performed better than the TPSs in 47% of the spine phantom cases. However, the DRS was more accurate in 93% of failing spine phantom cases (with an average improvement of 2.35%), indicating a deficiency in the institution's treatment planning system. Deficiencies in dose calculation accounted for 60% of the overall discrepancy between measured and planned doses among spine phantoms. In contrast, lung phantom DRS calculations were more accurate in only 35% and 42% of all and failing lung phantom cases respectively, indicating that dose calculation errors were not substantially present. These errors accounted for only 30% of the overall discrepancy between measured and planned doses.

CONCLUSIONS

Dose calculation errors are common and substantial in IROC spine phantom irradiations, highlighting a major failure mode in this phantom and in clinical treatment management of these cases. In contrast, dose calculation accuracy had only a minimal contribution to failing lung phantom results, indicating that other failure modes drive problems with this phantom and similar clinical treatments.

A comparison of IROC and ACDS on-site audits of reference and non-reference dosimetry

PURPOSE

Consistency between different international quality assurance groups is important in the progress toward similar standards and expectations in radiotherapy dosimetry around the world, and in the context of consistent clinical trial data from international trial participants. This study compares the dosimetry audit methodology and results of two international quality assurance groups performing a side-by-side comparison at the same radiotherapy department, and interrogates the ability of the audits to detect deliberately introduced errors.

METHODS

A comparison of the core dosimetry components of reference and non-reference audits was conducted by the Imaging and Radiation Oncology Core (IROC, Houston, USA) and the Australian Clinical Dosimetry Service (ACDS, Melbourne, Australia). A set of measurements were conducted over 2 days at an Australian radiation therapy facility in Melbourne. Each group evaluated the reference dosimetry, output factors, small field output factors, percentage depth dose (PDD), wedge, and off-axis factors according to their standard protocols. IROC additionally investigated the Electron PDD and the ACDS investigated the effect of heterogeneities. In order to evaluate and compare the performance of these audits under suboptimal conditions, artificial errors in percentage depth dose (PDD), EDW, and small field output factors were introduced into the 6 MV beam model to simulate potential commissioning/modeling errors and both audits were tested for their sensitivity in detecting these errors.

RESULTS

With the plans from the clinical beam model, almost all results were within tolerance and at an optimal pass level. Good consistency was found between the two audits as almost all findings were consistent between them. Only two results were different between the results of IROC and the ACDS. The measurements of reference FFF photons showed a discrepancy of 0.7% between ACDS and IROC due to the inclusion of a 0.5% nonuniformity correction by the ACDS. The second difference between IROC and the ACDS was seen with the lung phantom. The asymmetric field behind lung measured by the ACDS was slightly (0.3%) above the ACDS's pass (optimal) level of 3.3%. IROC did not detect this issue because their measurements were all assessed in a homogeneous phantom. When errors were deliberately introduced neither audit was sensitive enough to pick up a 2% change to the small field output factors. The introduced PDD change was flagged by both audits. Similarly, the introduced error of using 25° wedge instead of 30° wedge was detectible in both audits as out of tolerance.

CONCLUSIONS

Despite different equipment, approach, and scope of measurements in on-site audits, there were clear similarities between the results from the two groups. This finding is encouraging in the context of a global harmonized approach to radiotherapy quality assurance and dosimetry audit.

Deep learning for patient-specific quality assurance: Identifying errors in radiotherapy delivery by radiomic analysis of gamma images with convolutional neural networks

PURPOSE

Patient-specific quality assurance (QA) for intensity-modulated radiation therapy (IMRT) is a ubiquitous clinical procedure, but conventional methods have often been criticized as being insensitive to errors or less effective than other common physics checks. Recently, there has been interest in the application of radiomics, quantitative extraction of image features, to radiotherapy QA. In this work, we investigate a deep learning approach to classify the presence or absence of introduced radiotherapy treatment delivery errors from patient-specific QA.

METHODS

Planar dose maps from 186 IMRT beams from 23 IMRT plans were evaluated. Each plan was transferred to a cylindrical phantom CT geometry. Three sets of planar doses were exported from each plan corresponding to (a) the error-free case, (b) a random multileaf collimator (MLC) error case, and (c) a systematic MLC error case. Each plan was delivered to the electronic portal imaging device (EPID), and planned and measured doses were used to calculate gamma images in an EPID dosimetry software package (for a total of 558 gamma images). Two radiomic approaches were used. In the first, a convolutional neural network with triplet learning was used to extract image features from the gamma images. In the second, a handcrafted approach using texture features was used. The resulting metrics from both approaches were input into four machine learning classifiers (support vector machines, multilayer perceptrons, decision trees, and k-nearest-neighbors) in order to determine whether images contained the introduced errors. Two experiments were considered: the two-class experiment classified images as error-free or containing any MLC error, and the three-class experiment classified images as error-free, containing a random MLC error, or containing a systematic MLC error. Additionally, threshold-based passing criteria were calculated for comparison.

RESULTS

In total, 303 gamma images were used for model training and 255 images were used for model testing. The highest classification accuracy was achieved with the deep learning approach, with a maximum accuracy of 77.3% in the two-class experiment and 64.3% in the three-class experiment. The performance of the handcrafted approach with texture features was lower, with a maximum accuracy of 66.3% in the two-class experiment and 53.7% in the three-class experiment. Variability between the results of the four machine learning classifiers was lower for the deep learning approach vs the texture feature approach. Both radiomic approaches were superior to threshold-based passing criteria.

CONCLUSIONS

Deep learning with convolutional neural networks can be used to classify the presence or absence of introduced radiotherapy treatment delivery errors from patient-specific gamma images. The performance of the deep learning network was superior to a handcrafted approach with texture features, and both radiomic approaches were better than threshold-based passing criteria. The results suggest that radiomic QA is a promising direction for clinical radiotherapy.

Remote beam output audits: a global assessment of results out of tolerance

Background and purpose

Remote beam output audits, which independently measure an institution’s machine calibration, are a common component of independent radiotherapy peer review. This work reviews the results and trends of these audit results across several organisations and geographical regions.

Materials and methods

Beam output audit results from the Australian Clinical Dosimetry Services, International Atomic Energy Agency, Imaging and Radiation Oncology Core, and Radiation Dosimetry Services were evaluated from 2010 to the present. The rate of audit results outside a ±5% tolerance was evaluated for photon and electron beams as a function of the year of irradiation and nominal beam energy. Additionally, examples of confirmed calibration errors were examined to provide guidance to clinical physicists and auditing bodies.

Results

Of the 210,167 audit results, 1323 (0.63%) were outside of tolerance. There was a clear trend of improved audit performance for more recent dates, and while all photon energies generally showed uniform rates of results out of tolerance, low (6 MeV) and high (≥18 MeV) energy electron beams showed significantly elevated rates. Twenty nine confirmed calibration errors were explored and attributed to a range of issues, such as equipment failures, errors in setup, and errors in performing the clinical reference calibration. Forty-two percent of these confirmed errors were detected during ongoing periodic monitoring, and not at the time of the first audit of the machine.

Conclusions

Remote beam output audits have identified, and continue to identify, numerous and often substantial beam calibration errors.

Error Detection in Intensity-Modulated Radiation Therapy Quality Assurance Using Radiomic Analysis of Gamma Distributions

PURPOSE

To improve the detection of errors in intensity-modulated radiation therapy (IMRT) with a novel method that uses quantitative image features from radiomics to analyze gamma distributions generated during patient specific quality assurance (QA).

METHODS AND MATERIALS

One hundred eighty-six IMRT beams from 23 patient treatments were delivered to a phantom and measured with electronic portal imaging device dosimetry. The treatments spanned a range of anatomic sites; half were head and neck treatments, and the other half were drawn from treatments for lung and rectal cancers, sarcoma, and glioblastoma. Planar gamma distributions, or gamma images, were calculated for each beam using the measured dose and calculated doses from the 3-dimensional treatment planning system under various scenarios: a plan without errors and plans with either simulated random or systematic multileaf collimator mispositioning errors. The gamma images were randomly divided into 2 sets: a training set for model development and testing set for validation. Radiomic features were calculated for each gamma image. Error detection models were developed by training logistic regression models on these radiomic features. The models were applied to the testing set to quantify their predictive utility, determined by calculating the area under the curve (AUC) of the receiver operator characteristic curve, and were compared with traditional threshold-based gamma analysis.

RESULTS

The AUC of the random multileaf collimator mispositioning model on the testing set was 0.761 compared with 0.512 for threshold-based gamma analysis. The AUC for the systematic mispositioning model was 0.717 versus 0.660 for threshold-based gamma analysis. Furthermore, the models could discriminate between the 2 types of errors simulated here, exhibiting AUCs of approximately 0.5 (equivalent to random guessing) when applied to the error they were not designed to detect.

CONCLUSIONS

The feasibility of error detection in patient-specific IMRT QA using radiomic analysis of QA images has been demonstrated. This methodology represents a substantial step forward for IMRT QA with improved sensitivity and specificity over current QA methods and the potential to distinguish between different types of errors.

Reference dosimetry data and modeling challenges for Elekta accelerators based on IROC-Houston site visit data

PURPOSE

Reference dosimetry data can provide an independent second check of acquired values when commissioning or validating a treatment planning system (TPS). The Imaging and Radiation Oncology Core at Houston (IROC-Houston) has measured numerous linear accelerators throughout its existence. The results of those measurements are given here, comparing accelerators and the agreement of measurement versus institutional TPS calculations.

METHODS

Data from IROC-Houston on-site reviews from 2000 through 2014 were analyzed for all Elekta accelerators, approximately 50. For each, consistent point dose measurements were conducted for several basic parameters in a water phantom, including percentage depth dose, output factors, small-field output factors, off-axis factors, and wedge factors. The results were compared by accelerator type independently for 6, 10, 15, and 18 MV. Distributions of the measurements for each parameter are given, providing the mean and standard deviation. Each accelerator's measurements were also compared to its corresponding TPS calculation from the institution to determine the level of agreement, as well as determining which dosimetric parameters were most often in error.

RESULTS

Accelerators were grouped by head type and reference dosimetric values were compiled. No class of linac had better overall agreement with its TPS, but percentage depth dose and output factors commonly agreed well, while small-field output factors, off-axis factors, and wedge factors often disagreed substantially from their TPS calculations.

CONCLUSION

Reference data has been collected and analyzed for numerous Elekta linacs, which provide an independent way for a physicist to double-check their own measurements to prevent gross treatment errors. In addition, treatment planning parameters more often in error have been highlighted, providing practical caution for physicists commissioning treatment planning systems for Elekta linacs.

Dosimetric end-to-end tests in a national audit of 3D conformal radiotherapy

BACKGROUND AND PURPOSE

Independent dosimetry audits improve quality and safety of radiation therapy. This work reports on design and findings of a comprehensive 3D conformal radiotherapy (3D-CRT) Level III audit.

MATERIALS AND METHODS

The audit was conducted as onsite audit using an anthropomorphic thorax phantom in an end-to-end test by the Australian Clinical Dosimetry Service (ACDS). Absolute dose point measurements were performed with Farmer-type ionization chambers. The audited treatment plans included open and half blocked fields, wedges and lung inhomogeneities. Audit results were determined as Pass Optimal Level (deviations within 3.3%), Pass Action Level (greater than 3.3% but within 5%) and Out of Tolerance (beyond 5%), as well as Reported Not Scored (RNS). The audit has been performed between July 2012 and January 2018 on 94 occasions, covering approximately 90% of all Australian facilities.

RESULTS

The audit pass rate was 87% (53% optimal). Fifty recommendations were given, mainly related to planning system commissioning. Dose overestimation behind low density inhomogeneities by the analytical anisotropic algorithm (AAA) was identified across facilities and found to extend to beam setups which resemble a typical breast cancer treatment beam placement. RNS measurements inside lung showed a variation in the opposite direction: AAA under-dosed a target beyond lung and over-dosed the lung upstream and downstream of the target. Results also highlighted shortcomings of some superposition and convolution algorithms in modelling large angle wedges.

CONCLUSIONS

This audit showed that 3D-CRT dosimetry audits remain relevant and can identify fundamental global and local problems that also affect advanced treatments.

Novel methodologies for dosimetry audits: Adapting to advanced radiotherapy techniques

With new radiotherapy techniques, treatment delivery is becoming more complex and accordingly, these treatment techniques require dosimetry audits to test advanced aspects of the delivery to ensure best practice and safe patient treatment. This review of novel methodologies for dosimetry audits for advanced radiotherapy techniques includes recent developments and future techniques to be applied in dosimetry audits. Phantom-based methods (i.e. phantom-detector combinations) including independent audit equipment and local measurement equipment as well as phantom-less methods (i.e. portal dosimetry, transmission detectors and log files) are presented and discussed. Methodologies for both conventional linear accelerator (linacs) and new types of delivery units, i.e. Tomotherapy, stereotactic devices and MR-linacs, are reviewed. Novel dosimetry audit techniques such as portal dosimetry or log file evaluation have the potential to allow parallel auditing (i.e. performing an audit at multiple institutions at the same time), automation of data analysis and evaluation of multiple steps of the radiotherapy treatment chain. These methods could also significantly reduce the time needed for audit and increase the information gained. However, to maximise the potential, further development and harmonisation of dosimetry audit techniques are required before these novel methodologies can be applied.