Application and Challenges of Statistical Process Control in Radiation Therapy Quality Assurance

Quality assurance (QA) is important for ensuring precision in radiation therapy. The complexity and resource-intensive nature of QA has increased with the continual evolution of equipment and techniques. An effective approach is to improve the process control technology and resource optimization. Statistical process control is an economical and efficient tool that has been widely used to monitor, control, and improve quality management processes and is now being increasingly used for radiation therapy QA. This article reviews the development and methodology of statistical process control technology, evaluates its suitability in radiation therapy QA practices, and assesses its importance and challenges in optimizing radiation therapy QA processes.

Automated contouring and statistical process control for plan quality in a breast clinical trial

Background and purpose

Automatic review of breast plan quality for clinical trials is time-consuming and has some unique challenges due to the lack of target contours for some planning techniques. We propose using an auto-contouring model and statistical process control to independently assess planning consistency in retrospective data from a breast radiotherapy clinical trial.

Materials and methods

A deep learning auto-contouring model was created and tested quantitatively and qualitatively on 104 post-lumpectomy patients' computed tomography images (nnUNet; train/test: 80/20). The auto-contouring model was then applied to 127 patients enrolled in a clinical trial. Statistical process control was used to assess the consistency of the mean dose to auto-contours between plans and treatment modalities by setting control limits within three standard deviations of the data's mean. Two physicians reviewed plans outside the limits for possible planning inconsistencies.

Results

Mean Dice similarity coefficients comparing manual and auto-contours was above 0.7 for breast clinical target volume, supraclavicular and internal mammary nodes. Two radiation oncologists scored 95% of contours as clinically acceptable. The mean dose in the clinical trial plans was more variable for lymph node auto-contours than for breast, with a narrower distribution for volumetric modulated arc therapy than for 3D conformal treatment, requiring distinct control limits. Five plans (5%) were flagged and reviewed by physicians: one required editing, two had clinically acceptable variations in planning, and two had poor auto-contouring.

Conclusions

An automated contouring model in a statistical process control framework was appropriate for assessing planning consistency in a breast radiotherapy clinical trial.

Guaranteed performance of individual control chart used in gamma passing rate-based patient-specific quality assurance

PURPOSE

To assess the effect of sampling variability on the performance of individual charts (I-charts) for PSQA and provide a robust and reliable method for unknown PSQA processes.

MATERIALS AND METHODS

A total of 1327 pretreatment PSQAs were analyzed. Different datasets with samples in the range of 20-1000 were used to estimate the lower control limit (LCL). Based on the iterative "Identify-Eliminate-Recalculate" and direct calculation without any outlier filtering procedures, five I-charts methods, namely the Shewhart, quantile, scaled weighted variance (SWV), weighted standard deviation (WSD), and skewness correction (SC) method, were used to compute the LCL. The average run length (ARL₀) and false alarm rate (FAR₀) were calculated to evaluate the performance of LCL.

RESULTS

The ground truth of the values of LCL, FAR₀, and ARL₀ obtained via in-control PSQAs were 92.31%, 0.135%, and 740.7, respectively. Further, for in-control PSQAs, the width of the 95% confidence interval of LCL values for all methods tended to decrease with the increase in sample size. In all sample ranges of in-control PSQAs, only the median LCL and ARL₀ values obtained via WSD and SWV methods were close to the ground truth. For the actual unknown PSQAs, based on the "Identify-Eliminate-Recalculate" procedure, only the median LCL values obtained by the WSD method were closest to the ground truth.

CONCLUSIONS

Sampling variability seriously affected the I-chart performance in PSQA processes, particularly for small samples. For unknown PSQAs, the WSD method based on the implementation of the iterative "Identify-Eliminate-Recalculate" procedure exhibited sufficient robustness and reliability.

Impact of electronic portal image quality on Elekta AQUA® collimator isocenter

PURPOSE

The collimator radiation isocenter position determined in AQUA^® v3.0 [Elekta AB, Stockholm, Sweden] software test "MLC Leaf and Jaw Position" was independently validated using an in-house MATLAB [Natick, MA: The MathWorks Inc.] script.

METHODS

The AQUA test determines radiation isocenter using the mean field center of nine 4 cm × 4 cm electronic portal imager (EPID) exposures at equidistant collimator angles. Impact of EPID image quality on AQUA reported isocenter for thirteen Elekta linear accelerators with Agility MLC heads were evaluated.

RESULTS

Of the thirteen, three had visually and quantitatively identifiable artifacts. For the ten good EPID's there was a systematic 0.25 mm offset of the MATLAB calculated mean field center relative to AQUA in the X-axis and Y-axis. This corresponds to one image pixel and was found to be due to differences in software co-ordinate convention. After subtracting this offset there was no significant difference in AQUA and MATLAB calculated isocenter.

CONCLUSIONS

For the three machines with poor image quality there was a demonstrated variation in AQUA calculated field center and therefore radiation isocenter relative to MATLAB. Restricting the region of interest (ROI) in AQUA software to only the irradiated section of the EPID brought AQUA and MATLAB result for these three machines into agreement.

Application of failure mode and effects analysis to validate a novel hybrid Linac QC program that integrates automated and conventional QC testing

A hybrid quality control (QC) program was developed that integrates automated and conventional Linac QC, realizing the benefits of both automated and conventional QC, increasing efficiency and maintaining independent measurement methods. Failure mode and effects analysis (FMEA) was then applied in order to validate the program prior to clinical implementation. The hybrid QC program consists of automated QC with machine performance check and DailyQA3 array on the TrueBeam Linac, and Delta4 volumetric modulated arc therapy (VMAT) standard plan measurements, alongside conventional monthly QC at a reduced frequency. The FMEA followed the method outlined in TG-100. Process maps were created for each treatment type at our center: VMAT, stereotactic body radiotherapy (SBRT), conformal, and palliative. Possible failure modes were established by evaluating each stage in the process map. The FMEA followed semiquantitative methods, using data from our QC records from eight Linacs over 3 years for the occurrence estimates, and simulation of failure modes in the treatment planning system, with scoring surveys for severity and detectability. The risk priority number (RPN) was calculated from the product of the occurrence, severity, and detectability scores and then normalized to the maximum and ranked to determine the most critical failure modes. The highest normalized RPN values (100, 90) were found to be for MLC position dynamic for both VMAT and SBRT treatments. The next highest score was 35 for beam position for SBRT, and the majority of scores were less than 20. Overall, these RPN scores for the hybrid Linac QC program indicated that it would be acceptable, but the high RPN score associated with the dynamic MLC failure mode indicates that it would be valuable to perform more rigorous testing of the MLC. The FMEA proved to be a useful tool in validating hybrid QC.

Statistical process control to monitor use of a web-based autoplanning tool

PURPOSE

To investigate the use of statistical process control (SPC) for quality assurance of an integrated web-based autoplanning tool, Radiation Planning Assistant (RPA).

METHODS

Automatically generated plans were downloaded and imported into two treatment planning systems (TPSs), RayStation and Eclipse, in which they were recalculated using fixed monitor units. The recalculated plans were then uploaded back to the RPA, and the mean dose differences for each contour between the original RPA and the TPSs plans were calculated. SPC was used to characterize the RPA plans in terms of two comparisons: RayStation TPS versus RPA and Eclipse TPS versus RPA for three anatomical sites, and variations in the machine parameters dosimetric leaf gap (DLG) and multileaf collimator transmission factor (MLC-TF) for two algorithms (Analytical Anisotropic Algorithm [AAA]) and Acuros in the Eclipse TPS. Overall, SPC was used to monitor the process of the RPA, while clinics would still perform their routine patient-specific QA.

RESULTS

For RayStation, the average mean percent dose differences across all contours were 0.65% ± 1.05%, -2.09% ± 0.56%, and 0.28% ± 0.98% and average control limit ranges were 1.89% ± 1.32%, 2.16% ± 1.31%, and 2.65% ± 1.89% for the head and neck, cervix, and chest wall, respectively. In contrast, Eclipse's average mean percent dose differences across all contours were -0.62% ± 0.34%, 0.32% ± 0.23%, and -0.91% ± 0.98%, while average control limit ranges were 1.09% ± 0.77%, 3.69% ± 2.67%, 2.73% ± 1.86%, respectively. Averaging all contours and removing outliers, a 0% dose difference corresponded with a DLG value of 0.202 ± 0.019 cm and MLC-TF value of 0.020 ± 0.001 for Acuros and a DLG value of 0.135 ± 0.031 cm and MLC-TF value of 0.015 ± 0.001 for AAA.

CONCLUSIONS

Differences in mean dose and control limits between RPA and two separately commissioned TPSs were determined. With varying control limits and means, SPC provides a flexible and useful process quality assurance tool for monitoring a complex automated system such as the RPA.

Variation in Elekta iView electronic portal imager pixel scale factor with gantry angle, and impact on multi-leaf collimator quality assurance

For Elekta Agility linear accelerators, the iViewGT electronic portal imaging device (EPID) is positioned at a nominal X‐Ray source‐to‐panel distance of 1600 mm. For display, image registration, and data processing purposes, the image pixels are scaled to spatial units at the treatment isocenter plane. This is achieved by applying a pixel scaling factor (PSF). During this investigation, the dependence of the PSF at cardinal gantry angles was determined along with the resulting effects on the multi‐leaf collimator (MLC) quality assurance (QA) results for three linear accelerators (linacs). The PSF was found to vary by 0.0018–0.0022 mm/pixel during gantry rotation, which resulted in variations in the mean MLC reported error of up to 0.8 mm at 100 mm off‐axis with the gantry rotated to 180°. Measurement and application of a gantry angle–specific PSF is a simple process that can be implemented to improve the accuracy of EPID‐based MLC QA at cardinal gantry angles.

Assessment of Statistical Process Control Based DVH Action Levels for Systematic Multi-Leaf Collimator Errors in Cervical Cancer RapidArc Plans

Background

In the patient-specific quality assurance (QA), DVH is a critical clinically relevant parameter that is finally used to determine the safety and effectiveness of radiotherapy. However, a consensus on DVH-based action levels has not been reached yet. The aim of this study is to explore reasonable DVH-based action levels and optimal DVH metrics in detecting systematic MLC errors for cervical cancer RapidArc plans.

Methods

In this study, a total of 148 cervical cancer RapidArc plans were selected and measured with COMPASS 3D dosimetry system. Firstly, the patient-specific QA results of 110 RapidArc plans were retrospectively reviewed. Then, DVH-based action limits (AL) and tolerance limits (TL) were obtained by statistical process control. Secondly, systematic MLC errors were introduced in 20 RapidArc plans, generating 380 modified plans. Then, the dose difference (%DE) in DVH metrics between modified plans and original plans was extracted from measurement results. After that, the linear regression model was used to investigate the detection limits of DVH-based action levels between %DE and systematic MLC errors. Finally, a total of 180 test plans (including 162 error-introduced plans and 18 original plans) were prepared for validation. The error detection rate of DVH-based action levels was compared in different DVH metrics of 180 test plans.

Results

A linear correlation was found between systematic MLC errors and %DE in all DVH metrics. Based on linear regression model, the systematic MLC errors between -0.94 mm and 0.88 mm could be caught by the TL of PTV₉₅ ([-1.54%, 1.51%]), and the systematic MLC errors between -1.00 mm and 0.80 mm could also be caught by the TL of PTV_mean ([-2.06%, 0.38%]). In the validation, for original plans, PTV₉₅ showed the minimum error detection rate of 5.56%. For error-introduced plans with systematic MLC errors more than 1mm, PTV_mean showed the maximum error detection rate of 88.89%, and then was followed by PTV₉₅ (86.67%). All the TL of DVH metrics showed a poor error detection rate in identifying error-induced plans with systematic MLC errors less than 1mm.

Conclusion

In 3D quality assurance of cervical cancer RapidArc plans, process-based tolerance limits showed greater advantages in distinguishing plans introduced with systematic MLC errors more than 1mm, and reasonable DVH-based action levels can be acquired through statistical process control. During DVH-based verification, main focus should be on the DVH metrics of target volume. OARs in low-dose regions were found to have a relatively higher dose sensitivity to smaller systematic MLC errors, but may be accompanied with higher false error detection rate.

A Bayesian control chart based on the beta distribution for monitoring the two-dimensional gamma index pass rate in the context of patient-specific quality assurance

PURPOSE

In the context of quality assurance in intensity modulated radiation therapy (IMRT), the aim of this work was two-fold: (a) to show that the beta distribution characterizes the two-dimensional gamma index pass rate (GIPR), and that the quantiles of the distribution should be used in order to compute the control limit (CL) for the detection of abnormally low GIPR, and (b) to introduce a Bayesian control chart that allows calculation of CLs from the first measurement.

METHODS

In order to enable monitoring of the GIPR from the first measurement, we developed a Bayesian control chart based on the beta distribution, elaborated according to the following two steps: (a) an iterative bayesian inference approach without any prior information on the GIPR distribution was used at the start of monitoring and the CL was progressively updated; and (b) when sufficient in-control arcs had been recorded and the estimators of the parameters of the beta distribution were sufficiently accurate, the CL of the chart was fixed to a constant value corresponding to the quantile of the beta distribution. The clinical utility of this approach is illustrated through a real data case study: monitoring the GIPR of patients treated with a moving gantry IMRT technique RapidArc^TM on a Novalis TrueBeam STx (Varian Medical Systems) linear accelerator equipped with an aS1200 electronic portal imager device.

RESULTS

We showed that some commonly used distributions for monitoring GIPR in the literature, such as normal or logarithm transformation, are not appropriate. We compared the CLs of those solutions with the CL of our chart based on the BD (CL = 95.14%). The comparison revealed that the CL for the normal law (CL = 97.62%) generated too many false positives, and that the CL of the Logarithm transformation (CL = 83.74%) could fail to efficiently detect (i.e., sufficiently early on or faster) changes in the process.

CONCLUSIONS

Successful GIPR monitoring requires careful and rigorous application of well-established statistical concepts in the field of statistical process control. In this paper, we stress the importance of carefully analyzing the distribution of the monitored characteristic that is plotted on the control chart. We propose a Bayesian control chart that can be viewed as a practical solution for early implementation of GIPR monitoring, starting from the first arc. We demonstrate that beta distribution is a better method for characterizing the GIPR, and thus, the use of this approach is expected to improve patient-specific quality assurance plans in radiotherapy.

Evaluation of Elekta Agility multi-leaf collimator performance using statistical process control tools

Purpose

To evaluate the performance and stability of Elekta Agility multi‐leaf collimator (MLC) leaf positioning using a daily, automated quality control (QC) test based on megavoltage (MV) images in combination with statistical process control tools, and identify special causes of variations in performance.

Methods

Leaf positions were collected daily for 13 Elekta linear accelerators over 11‐37 months using the automated QC test, which analyzes 23 MV images to determine the location of MLC leaves relative to radiation isocenter. Leaf positioning stability was assessed using individual and moving range control charts. Specification levels of ±0.5, ±1, and ±1.5 mm were tested to determine positional accuracy. The durations between out‐of‐control and out‐of‐specification events were determined. Peaks in out‐of‐control leaf positions were identified and correlated to servicing events recorded for the whole duration of data collection.

Results

Mean leaf position error was −0.01 mm (range −1.3–1.6). Data stayed within ±1 mm specification for 457 days on average (range 3–838) and within ±1.5 mm for the entire date range. Measurements stayed within ±0.5 mm for 1 day on average (range 0–17); however, our MLC leaves were not calibrated to this level of accuracy. Leaf position varied little over time, as confirmed by tight individual (mean ±0.19 mm, range 0.09–0.43) and moving range (mean 0.23 mm, range 0.10–0.53) control limits. Due to sporadic out‐of‐control events, the mean in‐control duration was 2.8 days (range 1–28.5). A number of factors were found to contribute to leaf position errors and out‐of‐control behavior, including servicing events, beam spot motion, and image artifacts.

Conclusions

The Elekta Agility MLC model was found to perform with high stability, as evidenced by the tight control limits. The in‐specification durations support the current recommendation of monthly MLC QC tests with a ±1 mm tolerance. Future work is on‐going to determine if performance can be optimized further using high‐frequency QC test results to drive recalibration frequency.

Statistical process control for electron beam monitoring

PURPOSE

To assess the electron beam monitoring statistical process control (SPC) in linear accelerator (linac) daily quality control. We present a long-term record of our measurements and evaluate which SPC-led conditions are feasible for maintaining control.

METHODS

We retrieved our linac beam calibration, symmetry, and flatness daily records for all electron beam energies from January 2008 to December 2013, and retrospectively studied how SPC could have been applied and which of its features could be used in the future. A set of adjustment interventions designed to maintain these parameters under control was also simulated.

RESULTS

All phase I data was under control. The dose plots were characterized by rising trends followed by steep drops caused by our attempts to re-center the linac beam calibration. Where flatness and symmetry trends were detected they were less-well defined. The process capability ratios ranged from 1.6 to 9.3 at a 2% specification level. Simulated interventions ranged from 2% to 34% of the total number of measurement sessions. We also noted that if prospective SPC had been applied it would have met quality control specifications.

CONCLUSIONS

SPC can be used to assess the inherent variability of our electron beam monitoring system. It can also indicate whether a process is capable of maintaining electron parameters under control with respect to established specifications by using a daily checking device, but this is not practical unless a method to establish direct feedback from the device to the linac can be devised.