A virtual dosimetry audit - Towards transferability of gamma index analysis between clinical trial QA groups

A joint ESTRO and AAPM guideline for development, clinical validation and reporting of artificial intelligence models in radiation therapy

BACKGROUND AND PURPOSE

Artificial Intelligence (AI) models in radiation therapy are being developed with increasing pace. Despite this, the radiation therapy community has not widely adopted these models in clinical practice. A cohesive guideline on how to develop, report and clinically validate AI algorithms might help bridge this gap.

METHODS AND MATERIALS

A Delphi process with all co-authors was followed to determine which topics should be addressed in this comprehensive guideline. Separate sections of the guideline, including Statements, were written by subgroups of the authors and discussed with the whole group at several meetings. Statements were formulated and scored as highly recommended or recommended.

RESULTS

The following topics were found most relevant: Decision making, image analysis, volume segmentation, treatment planning, patient specific quality assurance of treatment delivery, adaptive treatment, outcome prediction, training, validation and testing of AI model parameters, model availability for others to verify, model quality assurance/updates and upgrades, ethics. Key references were given together with an outlook on current hurdles and possibilities to overcome these. 19 Statements were formulated.

CONCLUSION

A cohesive guideline has been written which addresses main topics regarding AI in radiation therapy. It will help to guide development, as well as transparent and consistent reporting and validation of new AI tools and facilitate adoption.

A virtual audit system for intensity-modulated radiation therapy credentialing in Japan Clinical Oncology Group clinical trials: A pilot study

PURPOSE

The Medical Physics Working Group of the Radiation Therapy Study Group at the Japan Clinical Oncology Group is currently developing a virtual audit system for intensity-modulated radiation therapy dosimetry credentialing. The target dosimeters include films and array detectors, such as ArcCHECK (Sun Nuclear Corporation, Melbourne, Florida, USA) and Delta4 (ScandiDos, Uppsala, Sweden). This pilot study investigated the feasibility of our virtual audit system using previously acquired data.

METHODS

We analyzed 46 films (32 and 14 in the axial and coronal planes, respectively) from 29 institutions. Global gamma analysis between measured and planned dose distributions used the following settings: 3%/3 mm criteria (the dose denominator was 2 Gy), 30% threshold dose, no scaling of the datasets, and 90% tolerance level. In addition, 21 datasets from nine institutions were obtained for array evaluation. Five institutions used ArcCHECK, while the others used Delta4. Global gamma analysis was performed with 3%/2 mm criteria (the dose denominator was the maximum calculated dose), 10% threshold dose, and 95% tolerance level. The film calibration and gamma analysis were conducted with in-house software developed using Python (version 3.9.2).

RESULTS

The means ± standard deviations of the gamma passing rates were 99.4 ± 1.5% (range, 92.8%-100%) and 99.2 ± 1.0% (range, 97.0%-100%) in the film and array evaluations, respectively.

CONCLUSION

This pilot study demonstrated the feasibility of virtual audits. The proposed virtual audit system will contribute to more efficient, cheaper, and more rapid trial credentialing than on-site and postal audits; however, the limitations should be considered when operating our virtual audit system.

2-Dimensional IMRT dose audit: An Indonesian multicenter study

Intensity modulated radiation therapy (IMRT) is an advanced technique in radiation therapy delivery. IMRT depends on the accuracy of the multileaf collimator during treatment. Hence, the actual dose distribution can deviate from the treatment planning system's calculation. This study aimed to perform a multicentre planar dosimetry audit of radiotherapy centres in Indonesia, using the structure sets from AAPM TG-119. The gamma index used to evaluate the dose distribution was 3%/3 mm and 3%/2 mm. We observed 100% gamma index passing rates mostly in the 3%/3 mm evaluations. The gamma index passing rates dropped in the 3%/2 mm analysis. Most of the radiotherapy centres participating in this audit satisfied each criterion's tolerance limit of the action level. This study may become a first result for the next multicenter IMRT audit by using a standardized protocol.

An overview of the medical-physics-related verification system for radiotherapy multicenter clinical trials by the Medical Physics Working Group in the Japan Clinical Oncology Group-Radiation Therapy Study Group

The Japan Clinical Oncology Group-Radiation Therapy Study Group (JCOG-RTSG) has initiated several multicenter clinical trials for high-precision radiotherapy, which are presently ongoing. When conducting multi-center clinical trials, a large difference in physical quantities, such as the absolute doses to the target and the organ at risk, as well as the irradiation localization accuracy, affects the treatment outcome. Therefore, the differences in the various physical quantities used in different institutions must be within an acceptable range for conducting multicenter clinical trials, and this must be verified with medical physics consideration. In 2011, Japan's first Medical Physics Working Group (MPWG) in the JCOG-RTSG was established to perform this medical-physics-related verification for multicenter clinical trials. We have developed an auditing method to verify the accuracy of the absolute dose and the irradiation localization. Subsequently, we credentialed the participating institutions in the JCOG multicenter clinical trials that were using stereotactic body radiotherapy (SBRT) for lungs, intensity-modulated radiotherapy (IMRT) and volumetric-modulated arc therapy (VMAT) for several disease sites, and proton beam therapy (PT) for the liver. From the verification results, accuracies of the absolute dose and the irradiation localization among the participating institutions of the multicenter clinical trial were assured, and the JCOG clinical trials could be initiated.

Exploring the gamma surface: A new method for visualising modulated radiotherapy quality assurance results

PURPOSE

This work presents a novel method of visualising the results of patient-specific quality assurance (QA) for modulated radiotherapy treatment plans, using a three-dimensional distribution of gamma pass rates, referred to as the "gamma surface". The method was developed to aid in comparing borderline and failing QA plans, and to better compare patient-specific QA results between departments.

METHODS

Gamma surface plots were created for a representative sample of situations encountered during patient-specific QA. To produce a gamma surface plot, for each QA result, gamma pass rates were plotted as a heat map, with dose difference on one axis and distance-to-agreement on the other. This involved the calculation of 100 × 100 gamma pass rates over a dose difference and distance-to-agreement grid. As examples, five 220 × 680 arrays of dose points from radiotherapy treatment plans were compared against measurement data consisting of 21 × 66 arrays of dose points spaced 10 mm apart.

RESULTS

The gamma surface plots facilitated the rapid evaluation of criteria combinations for each plan, clearly highlighting the difference between plans that are modelled and delivered well, and those that are not. Large scale features were also evident in each surface, hinting at potential over-modulation, systematic dose errors, and small or large scale areas of disagreement in the distributions.

CONCLUSIONS

Gamma surface plots are a useful tool for investigating QA failures and borderline results, and have the capacity to grant insights into treatment plan QA performance that may otherwise be missed.

Monte Carlo Calculation of the Energy Spectrum of a 6 MeV Electron Beam using PENetration and Energy Loss of Positrons and Electrons Code

Background:

The limited bibliographic existence of research works on the use of Monte Carlo simulation to determine the energy spectra of electron beams compared to the information available regarding photon beams is a scientific task that should be resolved.

Aims:

In this work, Monte Carlo simulation was performed through the PENELOPE code of the Sinergy Elekta accelerator head to obtain the spectrum of a 6 MeV electron beam and its characteristic dosimetric parameters.

Materials and Methods:

The central-axis energy spectrum and the percentage depth dose curve of a 6 MeV electron beam of an Elekta Synergy linear accelerator were obtained by using Monte Carlo PENELOPE code v2014. For this, the linear accelerator head geometry, electron applicators, and water phantom were simplified. Subsequently, the interaction process between the electron beam and head components was simulated in a time of 86.4x10⁴ s.

Results:

From this simulation, the energy spectrum at the linear accelerator exit window and the surface of the phantom was obtained, as well as the associated percentage depth dose curves. The validation of the Monte Carlo simulation was performed by comparing the simulated and the measured percentage depth dose curves via the gamma index criterion. Measured percentage depth- dose was determined by using a Markus electron ionization chamber, type T23343. Characteristic parameters of the beam related with the PDD curves such as the maximum dose depth (R₁₀₀), 90% dose depth (R₉₀), 90% dose depth or therapeutic range (R₈₅), half dose depth (R₅₀), practical range (R_p), maximum range (R_max), surface dose (D_s), normalized dose gradient (G₀) and photon contamination dose (D_x) were determined. Parameters related with the energy spectrum, namely, the most probable energy of electrons at the surface (E_p,0) and electron average energy (E– ₀) were also determined.

Conclusion:

It was demonstrated that PENELOPE is an attractive and accurate tool for the obtaining of dosimetric parameters of a medical linear accelerator since it can reliably reproduce important clinical data such as the energy spectrum, depth dose, and dose profile.

Survey results of 3D-CRT and IMRT quality assurance practice

PURPOSE

To create a snapshot of common practices for 3D-CRT and intensity-modulated radiation therapy (IMRT) QA through a large-scale survey and compare to TG-218 recommendations.

METHODS

A survey of 3D-CRT and IMRT QA was constructed at and distributed by the IROC-Houston QA center to all institutions monitored by IROC (n = 2,861). The first part of the survey asked about methods to check dose delivery for 3D-CRT. The bulk of the survey focused on IMRT QA, inquiring about treatment modalities, standard tools used to verify planned dose, how assessment of agreement is calculated and the comparison criteria used, and the strategies taken if QA fails.

RESULTS

The most common tools for dose verification were a 2D diode array (52.8%), point(s) measurement (39.0%), EPID (27.4%), and 2D ion chamber array (23.9%). When IMRT QA failed, the highest average rank strategy utilized was to remeasure with the same setup, which had an average position ranking of 1.1 with 90.4% of facilities employing this strategy. The second highest average ranked strategy was to move to a new calculation point and remeasure (54.9%); this had an average ranking of 2.1.

CONCLUSION

The survey provided a snapshot of the current state of dose verification for IMRT radiotherapy. The results showed variability in approaches and that work is still needed to unify and tighten criteria in the medical physics community, especially in reference to TG-218's recommendations.

Technical note: A modified gamma evaluation method for dose distribution comparisons

Purpose

In this work we have developed a novel method of dose distribution comparison, the inverse gamma (IG) evaluation, by modifying the commonly used gamma evaluation method.

Methods

The IG evaluation calculates the gamma criteria (dose difference criterion, ΔD, or distance‐to‐agreement criterion, Δd) that are needed to achieve a predefined pass rate or gamma agreement index (GAI). In‐house code for evaluating IG with a fixed ΔD of 3% was developed using Python (v3.5.2) and investigated using treatment plans and measurement data from 25 retrospective patient specific quality assurance tests (53 individual arcs).

Results

It was found that when the desired GAI was set to 95%, approximately three quarters of the arcs tested were able to achieve Δd within 1 mm (mean Δd: 0.7 ± 0.5 mm). The mean Δd required in order for all points to pass the gamma evaluation (i.e., GAI = 100%) was 4.5 ± 3.1 mm. The possibility of evaluating IG by fixing the Δd or ΔD/Δd, instead of fixing the ΔD at 3%, was also investigated.

Conclusion

The IG method and its indices have the potential to be implemented clinically to quantify the minimum dose and distance criteria based on a specified GAI. This method provides additional information to augment standard gamma evaluation results during patient specific quality assurance testing of individual treatment plans. The IG method also has the potential to be used in retrospective audits to determine an appropriate set of local gamma criteria and action levels based on a cohort of patient specific quality assurance plans.

Dosimetric Multicenter Planning Comparison Studies for Stereotactic Body Radiation Therapy: Methodology and Future Perspectives

In this review a summary of the published literature pertaining to the stereotactic body radiation therapy multiplanning comparison, data sharing strategies, and implementation of benchmark planning cases to improve the skills and knowledge of the participating centers was investigated. A total of 30 full-text articles were included. The studies were subdivided in 3 categories: multiplanning studies on dosimetric variability, planning harmonization before clinical trials, and technical and methodologic studies. The methodology used in the studies were critically analyzed to find common and original elements with the pros and cons. Multicenter planning studies have played a key role in improving treatment plan harmonization, treatment plan compliance, and even clinical practices. This review has highlighted that some fundamental steps should be taken to transform a simple treatment planning comparison study into a potential credentialing method for stereotactic body radiation therapy accreditation. In particular, prescription and general requirements should always be well defined; data analysis should be performed with independent dose volume histogram or dose calculations; quality score indices should be constructed; feedback and correction strategies should be provided; and a simple web-based collaboration platform should be used. The results reported clearly showed that a crowd-based replanning approach is a viable method for achieving harmonization and standardization of treatment planning among centers using different technologies.

An objective method to evaluate radiation dose distributions varying by three orders of magnitude

PURPOSE

Modern radiotherapy practices typically report the absorbed dose (D) within the 5% relative isodose volume (i.e., the therapeutic dose region) to an accuracy of 3%-5%. Gamma-index analysis, the most commonly used method to evaluate dosimetric accuracy, has low sensitivity to discrepancies that occur outside of this region. The objective of this study was to develop an evaluation method with high sensitivity across dose distributions spanning three orders of magnitude.

METHODS

We generalized the gamma index to include an additional criterion, the absolute absorbed dose difference, specifically for the low-dose region (i.e., D ≤ 5%). We also proposed a method to objectively select the appropriate magnitudes for relative-dose-difference, absolute-dose-difference, and distance-to-agreement criteria. We demonstrated the generalized gamma-index method by first finding the appropriate generalized gamma-index agreement criteria at an interval of specified passing rates. Next, we used the generalized gamma index to evaluate one-, two-, and three-dimensional absorbed dose distributions in a water-box phantom and voxelized patient geometry.

RESULTS

Generalized gamma-index passing rates for one-, two-, and three-dimensional dose distributions were 55.4%, 44.5%, and 8.9%, respectively. Traditional gamma-index passing rates were 100%, 97.8%, and 96.4%, respectively. These results reveal that the generalized method has adequate sensitivity in all regions (i.e., therapeutic and low dose). Additionally, the algorithmic determination of triplets of agreement criteria revealed that they are strong functions of the specified passing rate.

CONCLUSIONS

The major finding of this work is that the proposed method provides an objective evaluation of the agreement of dose distributions spanning three orders of magnitude. In particular, this generalized method correctly characterized dosimetric agreement in the low-dose region, which was not possible by traditional methods. The proposed algorithmic selection of agreement criteria decreased subjectivity and requirements of user judgment and skill. This method could find utility in a variety of applications including dose-algorithm development and translation.