Statistical process control and verifying positional accuracy of a cobra motion couch using step-wedge quality assurance tool

Application and Challenges of Statistical Process Control in Radiation Therapy Quality Assurance

Quality assurance (QA) is important for ensuring precision in radiation therapy. The complexity and resource-intensive nature of QA has increased with the continual evolution of equipment and techniques. An effective approach is to improve the process control technology and resource optimization. Statistical process control is an economical and efficient tool that has been widely used to monitor, control, and improve quality management processes and is now being increasingly used for radiation therapy QA. This article reviews the development and methodology of statistical process control technology, evaluates its suitability in radiation therapy QA practices, and assesses its importance and challenges in optimizing radiation therapy QA processes.

Monitoring Variations in the Use of Automated Contouring Software

PURPOSE

Our purpose was to identify variations in the clinical use of automatically generated contours that could be attributed to software error, off-label use, or automation bias.

METHODS AND MATERIALS

For 500 head and neck patients who were contoured by an in-house automated contouring system, Dice similarity coefficient and added path length were calculated between the contours generated by the automated system and the final contours after editing for clinical use. Statistical process control was used and control charts were generated with control limits at 3 standard deviations. Contours that exceeded the thresholds were investigated to determine the cause. Moving mean control plots were then generated to identify dosimetrists who were editing less over time, which could be indicative of automation bias.

RESULTS

Major contouring edits were flagged for: 1.0% brain, 3.1% brain stem, 3.5% left cochlea, 2.9% right cochlea, 4.8% esophagus, 4.1% left eye, 4.0% right eye, 2.2% left lens, 4.9% right lens, 2.5% mandible, 11% left optic nerve, 6.1% right optic nerve, 3.8% left parotid, 5.9% right parotid, and 3.0% of spinal cord contours. Identified causes of editing included unexpected patient positioning, deviation from standard clinical practice, and disagreement between dosimetrist preference and automated contouring style. A statistically significant (P < .05) difference was identified between the contour editing practice of dosimetrists, with 1 dosimetrist editing more across all organs at risk. Eighteen percent (27/150) of moving mean control plots created for 5 dosimetrists indicated the amount of contour editing was decreasing over time, possibly corresponding to automation bias.

CONCLUSIONS

The developed system was used to detect statistically significant edits caused by software error, unexpected clinical use, and automation bias. The increased ability to detect systematic errors that occur when editing automatically generated contours will improve the safety of the automatic treatment planning workflow.

IPEM topical report: results of a 2022 UK survey on the use of linac manufacturer integrated quality control (MIQC)

In recent years Radiotherapy linear accelerator (linac) vendors have developed their own integrated quality control (QC) systems. Such manufacturer-integrated-quality-control (MIQC) has the potential to improve both the quality and efficiency of linac QC but is currently being developed and utilised in the absence of specific best-practice guidance. An Institute of Physics and Engineering in Medicine working party was commissioned with a view to develop guidance for the commissioning and implementation of MIQC. This study is based upon a survey of United Kingdom (UK) radiotherapy departments performed by the working party. The survey was distributed to all heads of radiotherapy physics in the UK and investigated availability and uptake, community beliefs and opinions, utilisation, user experience and associated procedures. The survey achieved a 95% response rate and demonstrated strong support (>95%) for its use and further development. MIQC systems are available in 79% of respondents' centres, and are in clinical use in 66%. The most common MIQC system was Varian MPC, in clinical use in 58% of responding centres, with CyberKnife AQA\E2E in 11%, TomoTherapy TQA in 8% and no users of Elekta Machine QA. A majority of users found their MIQC to be easy to use, reliable, and had five or more years of experience. Most users reported occasions of discrepancy in results between MIQC and conventional testing, but the majority considered this acceptable, indicating a false reporting frequency of quarterly or less. MIQC has shown value in preventative maintenance and early detection of machine deviations. There were inconsistent approaches in the utilisation and commissioning tests performed. Fewer than half of users perform QC of MIQC. 45% of responders have modified their QC processes with the introduction of MIQC, via replacement of conventional tests or reduction in their frequency. Future guidance is recommended to assist in the implementation of MIQC.

Performance assessment of surface-guided radiation therapy and patient setup in head-and-neck and breast cancer patients based on statistical process control

PURPOSE

To assess the effectiveness of SGRT in clinical applications through statistical process control (SPC).

METHODS

Taking the patients' positioning through optical surface imaging (OSI) as a process, the average level of process execution was defined as the process mean. Setup errors detected by cone-beam computed tomography (CBCT) and OSI were extracted for head-and-neck cancer (HNC) and breast cancer patients. These data were used to construct individual and exponentially weighted moving average (EWMA) control charts to analyze outlier fractions and small process shifts from the process mean. Using the control charts and process capability indices derived from this process, the patient positioning-related OSI performance and setup error were analyzed for each patient.

RESULTS

Outlier fractions and small shifts from the process mean that are indicative of setup errors were found to be widely prevalent, with the outliers randomly distributed between fractions. A systematic error of up to 1.6 mm between the OSI and CBCT results was observed in all directions, indicating a significantly degraded OSI performance. Adjusting this systematic error for each patient using setup errors of the first five fractions could effectively mitigate these effects. Process capability analysis following adjustment for systematic error indicated that OSI performance was acceptable (process capability index C_pk = 1.0) for HNC patients but unacceptable (C_pk < 0.75) for breast cancer patients.

CONCLUSION

SPC is a powerful tool for detecting the outlier fractions and process changes. Our application of SPC to patient-specific evaluations validated the suitability of OSI in clinical applications involving patient positioning.

8+ Year Performance of the Gamma Knife Perfexion/Icon Patient Positioning System and Possibilities for Preemptive Fault Detection Using Statistical Process Control

BACKGROUND

The large fractional doses, steep dose gradients, and small targets found in intracranial radiosurgery require extremely low beam delivery uncertainty. In the case of Gamma Knife radiosurgery (GKRS), this includes minimizing patient positioning system (PPS) positioning uncertainty. Existing QA techniques are recipe based, and feature point in time pass/fail tolerances. However, modern treatment machines, including the Gamma Knife Perfexion/Icon systems, record extensive internal data in treatment logs. These data can be analyzed through statistical process control (SPC) methods which are designed to detect changes in process behavior. The purpose of this study was to characterize the long-term (8+ year) performance of a Perfexion/Icon unit and use SPC methods to determine if performance changes could be detected at levels lower than existing QA and internal manufacturer performance tolerances.

METHODS

In-house software was developed to parse Perfexion/Icon log-files and store relevant information on shot delivery in a relational database. A last-in, first-out (LIFO) queuing algorithm was created to heuristically match messages associated with a given delivered shot. Filtering criteria were developed to filter QA and uncompleted shots. The resulting matched shots were extracted. Achieved versus planned PPS position was determined for each PPS motor as well as for the vector magnitude difference in PPS position. Exponentially weighted moving average (EWMA) control charts were plotted to determine when process behavior changed over time.

RESULTS

53833 shots were delivered over an 8+ year span in the study. The mean vector magnitude PPS difference was 32.7 µm, with 97.5% of all shots within 70.1 µm. Several changes in PPS positioning behavior were observed over time, corresponding with control system faults on several occasions requiring PPS recalibration. EWMA control charts clearly demonstrate that these faults could be identified and possibly predicted as many as 3 years before there were faults beyond control system tolerance.

CONCLUSION

The PPS of Gamma Knife Perfexion/Icon systems has extremely low positioning uncertainties. EWMA control chart method can be utilized to track PPS performance over time and can potentially detect changes in performance that may indicate a component requiring maintenance. This would allow planned service visits to mitigate problems and prevent unplanned downtime.

A patient risk model to determine the optimal output constancy check frequency for a radiotherapy machine

PURPOSE

The output constancy check, a basic quality control (QC) item for radiotherapy machines, is performed daily according to suggestions in technical reports by experienced experts. In this study, a patient risk model was built to determine the optimal frequency of an output constancy check for a specific radiotherapy machine.

METHODS AND MATERIALS

The method was based on the patient risk model and comprised three steps: 1) the power function graph was used to select a proper QC rule and the average number of QC measurements per QC rule evaluation. 2) The optimal QC frequency was determined by the minimum integer value of expected patients treated between QC measurements. 3) The individual control chart (I-Chart) was used to evaluate the effectiveness of the model. The model was implemented on the output constancy check of a Tomotherapy machine.

RESULTS

The QC rule with the limits set to the mean ± 3 standard deviations and 5 measurements per QC were selected according to the power function graph. The optimal frequency was observed every 21 patients. The I-Chart showed that the optimal frequency detected the machine failure earlier compared to the conventional daily frequency. The model could monitor whether Tomotherapy machine was in good condition and predicted the time to adjust the machine.

CONCLUSIONS

The optimal output constancy check frequency of a radiotherapy machine is determined by the number of patients, which uses patient risk model. The optimal frequency is superior to the conventional daily frequency in identifying machine failure earlier.

Technical note: A modified gamma evaluation method for dose distribution comparisons

Purpose

In this work we have developed a novel method of dose distribution comparison, the inverse gamma (IG) evaluation, by modifying the commonly used gamma evaluation method.

Methods

The IG evaluation calculates the gamma criteria (dose difference criterion, ΔD, or distance‐to‐agreement criterion, Δd) that are needed to achieve a predefined pass rate or gamma agreement index (GAI). In‐house code for evaluating IG with a fixed ΔD of 3% was developed using Python (v3.5.2) and investigated using treatment plans and measurement data from 25 retrospective patient specific quality assurance tests (53 individual arcs).

Results

It was found that when the desired GAI was set to 95%, approximately three quarters of the arcs tested were able to achieve Δd within 1 mm (mean Δd: 0.7 ± 0.5 mm). The mean Δd required in order for all points to pass the gamma evaluation (i.e., GAI = 100%) was 4.5 ± 3.1 mm. The possibility of evaluating IG by fixing the Δd or ΔD/Δd, instead of fixing the ΔD at 3%, was also investigated.

Conclusion

The IG method and its indices have the potential to be implemented clinically to quantify the minimum dose and distance criteria based on a specified GAI. This method provides additional information to augment standard gamma evaluation results during patient specific quality assurance testing of individual treatment plans. The IG method also has the potential to be used in retrospective audits to determine an appropriate set of local gamma criteria and action levels based on a cohort of patient specific quality assurance plans.

Departmental action limits for TQA energy variations defined by means of statistical process control methods

The purpose of this study is to define departmental action limits for energy percentage variation measured by means of step-wedge helical Tomotherapy quality assurance module. Individual charts using the Statistical Process Control techniques have been used to identify retrospectively out-of-control situations ascribable to documented actions performed on the Tomotherapy system. Using the in-control data of our analysis process capability indices (cp, cpk, cpm and cpmk) are calculated in order to document the real working condition of the Tomotherapy system. Our findings indicate use of an action limit of 1.0% for energy percentage variation difference between the measured and reference output is a good working condition of a Tomotherapy system. cp and cpk indices are suggested as good indices that correctly report the system capability. A method for calculating and reporting Tomotherapy action limits for the integrated self-checking TQA energy check was shown in this study. SPC technique has proven to be efficient in defining departmental action limits from retrospective data for TQA energy measurements, hence optimally enabling corrective improvements in the process of quality assurance.

A multi-institutional evaluation of machine performance check system on treatment beam output and symmetry using statistical process control

BACKGROUND

The automated and integrated machine performance check (MPC) tool was verified against independent detectors to evaluate its beam uniformity and output detection abilities to consider it suitable for daily quality assurance (QA).

METHODS

Measurements were carried out on six linear accelerators (each located at six individual sites) using clinically available photon and electron energies for a period up to 12 months (n = 350). Daily constancy checks on beam symmetry and output were compared against independent devices such as the SNC Daily QA 3, PTW Farmer ionization chamber, and SNC field size QA phantom. MPC uniformity detection of beam symmetry adjustments was also assessed. Sensitivity of symmetry and output measurements were assessed using statistical process control (SPC) methods to derive tolerances for daily machine QA and baseline resets to account for drifts in output readings. I-charts were used to evaluate systematic and nonsystematic trends to improve error detection capabilities based on calculated upper and lower control levels (UCL/LCL) derived using standard deviations from the mean dataset.

RESULTS

This study investigated the vendor's method of uniformity detection. Calculated mean uniformity variations were within ± 0.5% of Daily QA 3 vertical symmetry measurements. Mean MPC output variations were within ± 1.5% of Daily QA 3 and ±0.5% of Farmer ionization chamber detected variations. SPC calculated UCL values were a measure of change observed in the output detected for both MPC and Daily QA 3.

CONCLUSIONS

Machine performance check was verified as a daily quality assurance tool to check machine output and symmetry while assessing against an independent detector on a weekly basis. MPC output detection can be improved by regular SPC-based trend analysis to measure drifts in the inherent device and control systematic and random variations thereby increasing confidence in its capabilities as a QA device. A 3-monthly MPC calibration assessment was recommended based on SPC capability and acceptability calculations.

Tomotherapy treatment site specific planning using statistical process control

BACKGROUND

This study investigated planned MLC distribution and treatment region specific plan parameters to recommend optimal delivery parameters based on statistical process techniques.

METHODS

A cohort of 28 head and neck, 19 pelvic and 23 brain pre-treatment plans were delivered on a helical tomotherapy system using 2.5 cm field width. Parameters such as gantry period, leaf open time (LOT), actual modulation factor, LOT sonogram, treatment duration and couch travel were investigated to derive optimal range for plans that passed acceptable delivery quality assurance. The results were compared against vendor recommendations and previous publications.

RESULTS

No correlation was observed between vendor recommended gantry period and percentage of minimum leaf open times. The range of gantry period (min-max) observed was 16-21 s for head and neck, 15-22 s for pelvis and 13-18 s for brain plans respectively. It was also noted that the highest percentage (average (X-) ± SD) of leaf open times for a minimum time of 100 ms was seen for brain plans (53.9 ± 9.2%) compared to its corresponding head and neck (34.5 ± 4.2%) and pelvic (32.0 ± 9.4%) plans respectively.

CONCLUSIONS

We have proposed that treatment site specific delivery parameters be used during planning that are based on the treatment centre and have detailed recommendations and limitations for the studied cohort. This may enable to improve efficiency of treatment deliveries by reducing inaccuracies in MLC distribution.