Patient safety improvements in radiation treatment through 5 years of incident learning

Causal analysis of radiotherapy safety incidents based on a hybrid model of HFACS and Bayesian network

Objective

This research investigates the role of human factors of all hierarchical levels in radiotherapy safety incidents and examines their interconnections.

Methods

Utilizing the human factor analysis and classification system (HFACS) and Bayesian network (BN) methodologies, we created a BN-HFACS model to comprehensively analyze human factors, integrating the hierarchical structure. We examined 81 radiotherapy incidents from the radiation oncology incident learning system (RO-ILS), conducting a qualitative analysis using HFACS. Subsequently, parametric learning was applied to the derived data, and the prior probabilities of human factors were calculated at each BN-HFACS model level. Finally, a sensitivity analysis was conducted to identify the human factors with the greatest influence on unsafe acts.

Results

The majority of safety incidents reported on RO-ILS were traced back to the treatment planning phase, with skill errors and habitual violations being the primary unsafe acts causing these incidents. The sensitivity analysis highlighted that the condition of the operators, personnel factors, and environmental factors significantly influenced the occurrence of incidents. Additionally, it underscored the importance of organizational climate and organizational process in triggering unsafe acts.

Conclusion

Our findings suggest a strong association between upper-level human factors and unsafe acts among radiotherapy incidents in RO-ILS. To enhance radiation therapy safety and reduce incidents, interventions targeting these key factors are recommended.

Automated contouring and statistical process control for plan quality in a breast clinical trial

Background and purpose

Automatic review of breast plan quality for clinical trials is time-consuming and has some unique challenges due to the lack of target contours for some planning techniques. We propose using an auto-contouring model and statistical process control to independently assess planning consistency in retrospective data from a breast radiotherapy clinical trial.

Materials and methods

A deep learning auto-contouring model was created and tested quantitatively and qualitatively on 104 post-lumpectomy patients' computed tomography images (nnUNet; train/test: 80/20). The auto-contouring model was then applied to 127 patients enrolled in a clinical trial. Statistical process control was used to assess the consistency of the mean dose to auto-contours between plans and treatment modalities by setting control limits within three standard deviations of the data's mean. Two physicians reviewed plans outside the limits for possible planning inconsistencies.

Results

Mean Dice similarity coefficients comparing manual and auto-contours was above 0.7 for breast clinical target volume, supraclavicular and internal mammary nodes. Two radiation oncologists scored 95% of contours as clinically acceptable. The mean dose in the clinical trial plans was more variable for lymph node auto-contours than for breast, with a narrower distribution for volumetric modulated arc therapy than for 3D conformal treatment, requiring distinct control limits. Five plans (5%) were flagged and reviewed by physicians: one required editing, two had clinically acceptable variations in planning, and two had poor auto-contouring.

Conclusions

An automated contouring model in a statistical process control framework was appropriate for assessing planning consistency in a breast radiotherapy clinical trial.

Safety in radiation oncology (SAFRON): Learning about incident causes and safety barriers in external beam radiotherapy

PURPOSE

Safety in Radiation Oncology (SAFRON) is a reporting and learning system on radiotherapy and radionuclide therapy incidents and near misses. The primary aim of this paper is to examine whether any discernible patterns exist in the causes of reported incidents and safety barriers within the SAFRON system concerning external beam radiotherapy.

METHODS AND MATERIALS

This study focuses on external beam radiotherapy incidents, reviewing 1685 reports since the inception of SAFRON until December 2021. Reports that did not identify causes of incidents and safety barriers were excluded from the final study population.

RESULTS

Simple two-dimensional radiotherapy or electron beam therapy were represented by 97 reports, three-dimensional conformal radiotherapy by 39 reports, modulated arc therapy by 12 reports, intensity modulated radiation therapy by 11 reports, stereotactic radiosurgery by 4 reports, and radiotherapy with protons or other particles by 1 report, while for 92 of them, no information on treatment method had been provided. Most of the reported incidents were minor incidents and were discovered by the radiation therapist. Inadequate direction/information in staff communication was the most frequently reported cause of incident, and regular independent chart check was the most common safety barrier.

CONCLUSIONS

The results indicate that the majority of incidents were reported by radiation therapists, and the majority of these incidents were classified as minor. Communication problems and failure to follow standards/procedures/practices were the most frequent causes of incidents. Furthermore, regular independent chart checking was the most frequently identified safety barrier.

Multi-Institutional Stereotactic Body Radiation Therapy Incident Learning: Evaluation of Safety Barriers Using a Human Factors Analysis and Classification System

OBJECTIVES

Stereotactic body radiation therapy (SBRT) can improve therapeutic ratios and patient convenience, but delivering higher doses per fraction increases the potential for patient harm. Incident learning systems (ILSs) are being increasingly adopted in radiation oncology to analyze reported events. This study used an ILS coupled with a Human Factor Analysis and Classification System (HFACS) and barriers management to investigate the origin and detection of SBRT events and to elucidate how safeguards can fail allowing errors to propagate through the treatment process.

METHODS

Reported SBRT events were reviewed using an in-house ILS at 4 institutions over 2014-2019. Each institution used a customized care path describing their SBRT processes, including designated safeguards to prevent error propagation. Incidents were assigned a severity score based on the American Association of Physicists in Medicine Task Group Report 275. An HFACS system analyzed failing safeguards.

RESULTS

One hundred sixty events were analyzed with 106 near misses (66.2%) and 54 incidents (33.8%). Fifty incidents were designated as low severity, with 4 considered medium severity. Incidents most often originated in the treatment planning stage (38.1%) and were caught during the pretreatment review and verification stage (37.5%) and treatment delivery stage (31.2%). An HFACS revealed that safeguard failures were attributed to human error (95.2%), routine violation (4.2%), and exceptional violation (0.5%) and driven by personnel factors 32.1% of the time, and operator condition also 32.1% of the time.

CONCLUSIONS

Improving communication and documentation, reducing time pressures, distractions, and high workload should guide proposed improvements to safeguards in radiation oncology.

Quality Improvement Process with Incident Learning Program Helped Reducing Transcriptional Errors on Telecobalt Due to Mismatched Parameters in Different Generations

Purpose

Higher frequency of transcriptional errors in the radiotherapy electronic charts for patients on telecobalt was noted. We describe the impact of the quality improvement (QI) initiative under the department's incident learning program (ILP).

Materials and Methods

The multidisciplinary quality team under ILP was formed to identify the root cause and introduce methods to reduce (smart goal) the current transcription error rate of 40% to <5% over 12 months. A root cause analysis including a fishbone diagram, Pareto chart, and action prioritization matrix was done to identify key drivers and interventions. Plan-Do-Study-Act (PDSA) Cycle strategy was undertaken. The primary outcome was percentage charts with transcriptional errors per month. The balancing measure was "new errors" due to interventions. All errors were identified and corrected before patient treatment.

Results

The average baseline error rate was 44.14%. The two key drivers identified were education of the workforce involved and mechanical synchronization of various machine parameters. PDSA cycle 1 consisted of an education program and sensitization of the staff, post which the error rates dropped to 5.4% (t-test P = 0.03). Post-PDSA cycle 2 (synchronization of machine parameters), 1, 3, and 6 months and 1 year, the error rates were sustained to 5%, 4%, 3%, and 4% (t-test P > 0.05) with no new additional errors.

Conclusions

With various generations of machines and technologies that are not synchronized, the proneness of transcription errors can be very high which can be identified and corrected with a typical QI process under ILP.

The impact of COVID-19 on a high-volume incident learning system: A retrospective analysis

PURPOSE

The purpose of this work was to assess how the coronavirus disease 2019 (COVID-19) pandemic impacted our incident learning system data and communicate the impact of a major exogenous event on radiation oncology clinical practice.

METHODS

Trends in our electronic incident reporting system were analyzed to ascertain the impact of the COVID-19 pandemic, including any direct clinical changes. Incident reports submitted in the 18 months prior to the pandemic (September 14, 2018 to March 13, 2020) and reports submitted during the first 18 months of the pandemic (March 14, 2020 to September 13, 2021) were compared. The incident reports include several data elements that were evaluated for trends between the two time periods, and statistical analysis was performed to compare the proportions of reports.

RESULTS

In the 18 months prior to COVID-19, 192 reports were submitted per 1000 planning tasks (n = 832 total). In the first 18 months of the pandemic, 147 reports per 1000 planning tasks were submitted (n = 601 total), a decrease of 23.4%. Statistical analysis revealed that there were no significant changes among the data elements between the pre- and during COVID-19 time periods. An analysis of the free-text narratives in the reports found that phrases related to pretreatment imaging were common before COVID-19 but not during. Conversely, phrases related to intravenous contrast, consent for computed tomography, and adaptive radiotherapy became common during COVID-19.

CONCLUSIONS

The data elements captured by our incident learning system were stable after the onset of the COVID-19 pandemic, with no statistically significant findings after correction for multiple comparisons. A trend toward fewer reports submitted for low-risk issues was observed. The methods used in the work can be generalized to events with a large-scale impact on the clinic or to monitor an incident learning system to drive future improvement activities.

Incident review in radiation oncology

By its very nature, radiation oncology is a complex, multi-profession dynamic modality of cancer treatment. There are multiple steps with many handovers of work and many opportunities for patient safety to be compromised. Patient safety events can manifest as either actual incidents or near miss/close call events. Reporting and learning from these events is key to quality improvement and patient safety. In this paper, we aim to provide an overview of radiation oncology incident reporting and learning systems. We review the importance of the use of a standardized taxonomy and classification that is specific to radiation oncology workflow, the international systems in current use and the current reporting requirements in Australia and New Zealand. Equally important is the culture that exists alongside the incident learning system. A just culture, where support for reporting exists and there is an adaptive responsive environment to learn and improve patient safety. The incident learning and patient safety system requires constant effort to make it a success. We describe potential measures of safety culture and of relative patient safety and recommend their routine use. We offer this review to stimulate the effort towards a binational voluntary incident learning system, a key pillar for the improvement in patient safety in radiation oncology.

The radiotherapy quality assurance gap among phase III cancer clinical trials

PURPOSE

Quality assurance (QA) practices improve the quality level of oncology trials by ensuring that the protocol is followed and the results are valid and reproducible. This study investigated the utilization of QA among randomized controlled trials that involve radiotherapy (RT).

METHODS AND MATERIALS

We searched ClinicalTrials.gov in February 2020 for all phase III oncology randomized clinical trials (RCTs). These trials were screened for RT-specific RCTs that had published primary trial results. Information regarding QA in each trial was collected from the study publications and trial protocol if available. Two individuals independently performed trial screening and data collection. Pearson's Chi-square tests analyses were used to assess factors that were associated with QA inclusion in RT trials.

RESULTS

Forty-two RCTs with RT as the primary intervention or as a mandatory component of the protocol were analyzed; the earliest was started in 1994 and one trial was still active though not recruiting. Twenty-nine (69%) trials mandated RT quality assurance (RTQA) practices as part of the trial protocol, with 19 (45%) trials requiring institutional credentialing. Twenty-one (50%) trials published protocol deviation outcomes. Clinical trials involving advanced radiation techniques (IMRT, VMAT, SRS, SBRT) did not include more RTQA than trials without these advanced techniques (73% vs. 65%, p = 0.55). Trials that reported protocol deviation outcomes were associated with mandating RTQA in their protocols as compared to trials that did not report these outcomes (100% vs. 38%, p < 0.001).

CONCLUSIONS

There is a lack of RTQA utilization and transparency in RT clinical trials. It is imperative for RT trials to include increased QA for safe, consistent, and high-quality RT planning and delivery.

Safety culture to improve accidental event reporting in radiotherapy

BACKGROUND AND PURPOSE

The potential for unintended and adverse radiation exposure in radiotherapy (RT) is real and should be studied because RT is a highly complex, multistep process, which requires input from numerous individuals from different areas and steps of the RT workflow. The 'Incident' (I) is an event the consequence of which is not negligible from the point of view of protection or safety. A 'near miss' (NM) is defined as an event that is highly likely to happen but did not occur. The purpose of this work is to show that through systematic reporting and analysis of these adverse events, their occurrence can be reduced.

MATERIALS AND METHODS

Staff were trained to report every type of unintended and adverse radiation exposure and to provide a full description of it.

RESULTS

By 2018, 110 worksheets had been collected, with an average of 6.1 adverse events per year (with 780 patients treated per year, meaning an average incident rate of 0.78%). In 2001-2009, 37 events were registered (13 I and 24 NM), the majority of them were in the decision phase (12/37), while in 2010-2013, there were 42 (1 I and 41 NM) in both the dose-calculation and transfer phase (19/42). In 2014-2018, 31 events (1 I and 30 NM) were equally distributed across the phases of the RT process. In 9/15 cases of I, some checkpoint was introduced.

CONCLUSION

The complexity of the RT workflow is prone to errors, and this must be taken into account by encouraging a safety culture. The aim of this paper is to present the collected incidents and near misses and to show how organization and practice were modified by the acquired knowledge.

Impact of technological and departmental changes on incident rates in radiation oncology over a seventeen-year period

INTRODUCTION

Advancements in technology and processes are designed to bring improvement. However, this is often achieved in parallel with increases in complexity, simultaneously presenting opportunities for new types of errors. This study aims to contextualise the impact of internal departmental changes upon radiation incidents and near misses recorded.

METHODS

A timeline of events and a comprehensive incident categorisation system were applied to all radiation incidents and near misses recorded at the Princess Alexandra Hospital Radiation Oncology department from 2003 to 2019, inclusive. Descriptive statistics were performed to identify the type and number of incidents reported during the time period in relation to potential changes within the department, with a focus on the implementation of an electronic environment.

RESULTS

Over the seventeen-year period, 157 incidents and 76 near misses were reported. The majority of incidents were classified as 'procedural' (78%), with 'treatment' being both the highest point of error and point of detection (49% and 85%, respectively). The largest number of incidents and near misses were reported in 2018 (n = 39) which was also a year that experienced the largest number of departmental changes (n = 16), including the move to a completely electronic planning process.

CONCLUSIONS

Changes within the department were followed by an increasing number of reported incidents. Proactive measures should be undertaken prior to the implementation of major changes within the department to aid in the minimisation of incident occurrence.

Differences in Safety Report Event Types Submitted by Graduate Medical Education Trainees Compared With Other Healthcare Team Members

OBJECTIVES

Graduate medical education (GME) trainees have a unique perspective from which to identify and report patient safety concerns. However, it is not known how safety reports submitted by GME trainees differ from those submitted by other clinical staff. We hypothesized that GME trainees were more likely to submit safety reports regarding transitions of care, delays in care, and lapses in communication, and reports of higher severity compared with other frontline staff such as nurses, pharmacists, and other providers.

METHODS

Patient safety reports submitted by clinical staff for 1 year at an academic tertiary care children's hospital were retrospectively reviewed and categorized by reporter type. Severity level and event type were analyzed by reporter type, and repeat χ2 tests were used to compare the percentage of reports at each severity level and in each event type submitted by GME trainees compared with each other reporter type.

RESULTS

Graduate medical education trainees submitted reports of greater severity (level E/F/G) compared with nurses (10% versus 5%, P = 0.021) and pharmacists (10% versus 2%, P = 0.001). A greater percent of GME trainees' reports were categorized as errors in transitions of care, diagnosis, ordering, laboratory collection, and care delays compared with several other reporter types.

CONCLUSIONS

Graduate medical education trainees identify system vulnerabilities not detected by other personnel, supporting efforts to increase safety reporting by GME trainees.

Failure mode and effect analysis for linear accelerator-based paraspinal stereotactic body radiotherapy

Introduction

Paraspinal stereotactic body radiotherapy (SBRT) involves risks of severe complications. We evaluated the safety of the paraspinal SBRT program in a large academic hospital by applying failure modes and effects analysis.

Methods

The analysis was conducted by a multidisciplinary committee (two therapists, one dosimetrist, four physicists, and two radiation oncologists). The paraspinal SBRT workflow was segmented into four phases (simulation, treatment planning, delivery, and machine quality assurance (QA)). Each phase was further divided into a sequence of sub‐processes. Potential failure modes (PFM) were identified from each subprocess and scored in terms of the frequency of occurrence, severity and detectability, and a risk priority number (RPN). High‐risk PFMs were identified based on RPN and were studied for root causes using fault tree analysis.

Results

Our paraspinal SBRT process was characterized by eight simulations, 11 treatment planning, nine delivery, and two machine QA sub‐processes. There were 18, 29, 19, and eight PFMs identified from simulation, planning, treatment, and machine QA, respectively. The median RPN of the PFMs was 62.9 for simulation, 68.3 for planning, 52.9 for delivery, and 22.0 for machine QA. The three PFMs with the highest RPN were: previous radiotherapy outside the institution is not accurately evaluated (RPN: 293.3), incorrect registration between diagnostic magnetic resonance imaging and simulation computed tomography causing incorrect contours (273.0), and undetected patient movement before ExacTrac baseline (217.8). Remedies to the high RPN failures were implemented, including staff education, standardized magnetic resonance imaging acquisition parameters, and an image fusion process, and additional QA on beam steering.

Conclusions

A paraspinal SBRT workflow in a large clinic was evaluated using a multidisciplinary and systematic risk analysis, which led to feasible solutions to key root causes. Treatment planning was a major source of PFMs that systematically affect the safety and quality of treatments. Accurate evaluation of external treatment records remains a challenge.

Improving Incident Reporting in a Hospital-Based Radiation Oncology Department: The Impact of a Customized Crew Resource Training and Event Reporting Intervention

Background

Radiation oncology (RO) is a high-risk environment with an increased potential for error due to the complex automated and manual interactions between heterogeneous teams and advanced technologies. Errors involving procedural deviations­­ can adversely impact patient morbidity and mortality. Under-reporting of errors is common in healthcare for reasons such as fear of retribution, liability, embarrassment, etc. Incident reporting is a proven tool for learning from errors and, when effectively implemented, can improve quality and safety. Crew resource management (CRM) employs just culture principles with a team-based safety system. The pillars of CRM include mandatory error reporting and structured training to proactively identify, learn from, and mitigate incidents. High-reliability organizations, such as commercial aviation, have achieved exemplary safety performance since adopting CRM strategies.

Objective

Our aim was to double the rate of staff error reporting from baseline rates utilizing CRM strategies during a six-month study period in a hospital-based radiation oncology (RO) department.

Methods

This quasi-experimental study involved a retrospective review of reported radiation oncology incidents between January 2015 and March 2016, which helped inform the development and implementation of a two-step custom CRM training and incident learning system (ILS) intervention in May 2016. A convenience sample of approximately 50 RO staff (Staff) performing over 100 external beam and daily brachytherapy treatments participated in weekly training for six months while continuing to report errors on a hospital-enterprise system. A discipline-specific incident learning system (ILS) customized for the department was added during the last three months of the study, enabling staff to identify, characterize, and report incidents and potential errors. Weekly process control charts used to trend incident reporting rates (total number of reported incidents in a given month /1000 fractions), and custom reports characterizing the potential severity as well as the location of incidents along the treatment path, were reviewed, analyzed, and addressed by an RO multidisciplinary project committee established for this study.

Results

A five-fold increase in the monthly reported number of incidents (n = 9.3) was observed during the six-month intervention period as compared to the 16-month pre-intervention period (n = 1.8). A significant increase (>3 sigma) was observed when the custom reporting system was added during the last three study months.

Conclusion

A discipline-specific electronic ILS enabling the characterization of individual RO incidents combined with routine CRM training is an effective method for increasing staff incident reporting and engagement, leading to a more systematic, team-based mitigation process. These combined strategies allowed for real-time reporting, analysis, and learning that can be used to enhance patient safety, improve teamwork, streamline communication, and advance a culture of safety.

Improving radiotherapy safety and efficiency with the customized ARIA oncology information system

OBJECTIVE

To improve safety and efficiency of radiotherapy process by customizing a Varian ARIA oncology information system following the guidelines provided in AAPM TG-100 report.

METHODS

First, failure mode and effects analysis (FMEA) and quality management program were implemented for radiotherapy process. We have customized the visual care path in the ARIA system and set up a series of templates for simulation, prescription, contouring, treatment planning, and multiple checklists. Average time of activities' completion and amount of planning errors were compared before and after the use of the customized ARIA to evaluate its impact on the efficiency and safety of radiotherapy.

RESULTS

Completion time and on-time completion rate of the key activities in the care path are improved. The time of OAR/targets contouring decreases from (1.94±1.51) days to (1.64±1.07) days (p = 0.003), with the on-time completion rate increases from 77.4%to 83.3%(p = 0.048). Treatment planning time decreases from (0.81±0.65) days to (0.55±0.51) days (p < 0.001), with the on-time completion rate increases from 96.6%to 98.3%(p = 0.163). Waiting time of patients decreases from (4.50±1.83) days to (4.04±1.34) days (p < 0.001), with the on-time completion rate increases from 81.9%to 89.7%(p = 0.003). In addition, the average plan error rate decreases from 5.5%(2.9%for safety errors and 2.6%for non-normative errors) to 2.4%(1.6%for safety errors and 0.8%for non-normative errors) (p = 0.029).

CONCLUSION

Our study demonstrates that the customized ARIA system has the potential to promote efficiency and safety in radiotherapy process management. It is beneficial to organize and accelerate the treatment process with more effective communications and fewer errors.

Critical success factors for implementation of an incident learning system in radiation oncology department

Aim

The aim of this study was to analyze critical success factors (CSFs) for implementation of an incident learning system (ILS) in a radiation oncology department (ROD) and evaluate the perception of the staff members along this process.

Background

Implementing an ILS is a way to leverage learning from incidents and is a tool for improving patient safety, consisting of a cycle of reporting and analyzing events as well as taking preventive actions. ILS implementation is challenging, requiring specific resources and cultural changes.

Materials and methods

An ILS was designed and implemented based on the CSF identified in the literature review. Before starting the ILS implementation, a structured survey was applied to assess dimensions of patient safety culture. After the period of implementation (7 months), the survey was applied again and compared with the initial assessment, and interviews were performed with staff members to evaluate the overall satisfaction with ILS and CSFs.

Results

Statistically significant improvements were observed in 5 dimensions (12 totals) of the safety culture survey, considering time points before and after the ILS implementation. According to interviewees, "Facilitating committee", "Efficient data collection", "Focus on improvement", "Just culture" and "Feedback to users" were the most relevant CSFs.

Conclusions

The ILS designed and implemented at ROD was perceived as an important tool to support quality and safety initiatives, promoting the improvement in safety culture. The ILS implementation critical success factors were identified and have shown good agreement between the results of the literature and the users' practical perception.

Automatic Verification of Beam Apertures for Cervical Cancer Radiation Therapy

PURPOSE

Automated tools can help identify radiation treatment plans of unacceptable quality. To this end, we developed a quality verification technique to automatically verify the clinical acceptability of beam apertures for 4-field box treatments of patients with cervical cancer. By comparing the beam apertures to be used for treatment with a secondary set of beam apertures developed automatically, this quality verification technique can flag beam apertures that may need to be edited to be acceptable for treatment.

METHODS AND MATERIALS

The automated methodology for creating verification beam apertures uses a deep learning model trained on beam apertures and digitally reconstructed radiographs from 255 clinically acceptable planned treatments (as rated by physicians). These verification apertures were then compared with the treatment apertures using spatial comparison metrics to detect unacceptable treatment apertures. We tested the quality verification technique on beam apertures from 80 treatment plans. Each plan was rated by physicians, where 57 were rated clinically acceptable and 23 were rated clinically unacceptable.

RESULTS

Using various comparison metrics (the mean surface distance, Hausdorff distance, and Dice similarity coefficient) for the 2 sets of beam apertures, we found that treatment beam apertures rated acceptable had significantly better agreement with the verification beam apertures than those rated unacceptable (P < .01). Upon receiver operating characteristic analysis, we found the area under the curve for all metrics to be 0.89 to 0.95, which demonstrated the high sensitivity and specificity of our quality verification technique.

CONCLUSIONS

We found that our technique of automatically verifying the beam aperture is an effective tool for flagging potentially unacceptable beam apertures during the treatment plan review process. Accordingly, we will clinically deploy this quality verification technique as part of a fully automated treatment planning tool and automated plan quality assurance program.

Automated Radiation Treatment Planning for Cervical Cancer

The radiation treatment-planning process includes contouring, planning, and reviewing the final plan, and each component requires substantial time and effort from multiple experts. Automation of treatment planning can save time and reduce the cost of radiation treatment, and potentially provides more consistent and better quality plans. With the recent breakthroughs in computer hardware and artificial intelligence technology, automation methods for radiation treatment planning have achieved a clinically acceptable level of performance in general. At the same time, the automation process should be developed and evaluated independently for different disease sites and treatment techniques as they are unique from each other. In this article, we will discuss the current status of automated radiation treatment planning for cervical cancer for simple and complex plans and corresponding automated quality assurance methods. Furthermore, we will introduce Radiation Planning Assistant, a web-based system designed to fully automate treatment planning for cervical cancer and other treatment sites.

Quality management in radiation therapy: A 15 year review of incident reporting in two integrated cancer centres

•

Fifteen years of reported incidents were reviewed.

•

Overall reduction in incident severity overtime identified.

•

New technology associated to reduced incident severity.

•

Reporting culture associated to reporting rates.

•

Taxonomy changes required to improve ISL and incident classification.

Fifteen years of reported incidents were reviewed to provide insight into the effectiveness of an Incident Learning System (ISL). The actual error rate over the 15 years was 1.3 reported errors per 1000 treatment attendances. Incidents were reviewed using a regression model. The average number of incidents per year and the number of incidents per thousand attendances declined over time. Two seven-year periods were considered for analysis and the average for the first period (2005–2011) was 6 reported incidents per 1000 attendances compared to 2 incidents for the later period (2012–2018), p < 0.05. SAC 1 and SAC 2 errors have reduced over time and the reduction could be attributed to the quality assurance aspect of IGRT where the incident is identified prior to treatment delivery rather than after, reducing the severity of any potential incidents. The reasoning behind overall reduction in incident reporting over time is unclear but may be associated to quality and technology initiatives, issues with the ISL itself or a change in the staff reporting culture.

Optimizing efficiency and safety in external beam radiotherapy using automated plan check (APC) tool and six sigma methodology

PURPOSE

To develop and implement an automated plan check (APC) tool using a Six Sigma methodology with the aim of improving safety and efficiency in external beam radiotherapy.

METHODS

The Six Sigma define-measure-analyze-improve-control (DMAIC) framework was used by measuring defects stemming from treatment planning that were reported to the departmental incidence learning system (ILS). The common error pathways observed in the reported data were combined with our departmental physics plan check list, and AAPM TG-275 identified items. Prioritized by risk priority number (RPN) and severity values, the check items were added to the APC tool developed using Varian Eclipse Scripting Application Programming Interface (ESAPI). At 9 months post-APC implementation, the tool encompassed 89 check items, and its effectiveness was evaluated by comparing RPN values and rates of reported errors. To test the efficiency gains, physics plan check time and reported error rate were prospectively compared for 20 treatment plans.

RESULTS

The APC tool was successfully implemented for external beam plan checking. FMEA RPN ranking re-evaluation at 9 months post-APC demonstrated a statistically significant average decrease in RPN values from 129.2 to 83.7 (P < .05). After the introduction of APC, the average frequency of reported treatment-planning errors was reduced from 16.1% to 4.1%. For high-severity errors, the reduction was 82.7% for prescription/plan mismatches and 84.4% for incorrect shift note. The process shifted from 4σ to 5σ quality for isocenter-shift errors. The efficiency study showed a statistically significant decrease in plan check time (10.1 ± 7.3 min, P = .005) and decrease in errors propagating to physics plan check (80%).

CONCLUSIONS

Incorporation of APC tool has significantly reduced the error rate. The DMAIC framework can provide an iterative and robust workflow to improve the efficiency and quality of treatment planning procedure enabling a safer radiotherapy process.

A risk assessment of automated treatment planning and recommendations for clinical deployment

Purpose

To assess the risk of failure of a recently developed automated treatment planning tool, the radiation planning assistant (RPA), and to determine the reduction in these risks with implementation of a quality assurance (QA) program specifically designed for the RPA.

Methods

We used failure mode and effects analysis (FMEA) to assess the risk of the RPA. The steps involved in the workflow of planning a four‐field box treatment of cervical cancer with the RPA were identified. Then, the potential failure modes at each step and their causes were identified and scored according to their likelihood of occurrence, severity, and likelihood of going undetected. Additionally, the impact of the components of the QA program on the detectability of the failure modes was assessed. The QA program was designed to supplement a clinic's standard QA processes and consisted of three components: (a) automatic, independent verification of the results of automated planning; (b) automatic comparison of treatment parameters to expected values; and (c) guided manual checks of the treatment plan. A risk priority number (RPN) was calculated for each potential failure mode with and without use of the QA program.

Results

In the RPA automated treatment planning workflow, we identified 68 potential failure modes with 113 causes. The average RPN was 91 without the QA program and 68 with the QA program (maximum RPNs were 504 and 315, respectively). The reduction in RPN was due to an improvement in the likelihood of detecting failures, resulting in lower detectability scores. The top‐ranked failure modes included incorrect identification of the marked isocenter, inappropriate beam aperture definition, incorrect entry of the prescription into the RPA plan directive, and lack of a comprehensive plan review by the physician.

Conclusions

Using FMEA, we assessed the risks in the clinical deployment of an automated treatment planning workflow and showed that a specialized QA program for the RPA, which included automatic QA techniques, improved the detectability of failures, reducing this risk. However, some residual risks persisted, which were similar to those found in manual treatment planning, and human error remained a major cause of potential failures. Through the risk analysis process, we identified three key aspects of safe deployment of automated planning: (a) user training on potential failure modes; (b) comprehensive manual plan review by physicians and physicists; and (c) automated QA of the treatment plan.

Implementation and operation of incident learning across a newly-created health system

PURPOSE

The purpose of this work is to describe our experience launching an expanded incident learning system for patient safety and quality that takes into account aspects beyond therapeutic dose delivery, specifically imaging/simulation incidents, medical care incidents, and operational issues.

METHODS

Our ILS was designed for a newly created health system comprised of a midsized academic hospital and two smaller community hospitals. The main design goal was to create a highly sensitive system to capture as much information throughout the department as possible. Reports were classified according to incidents and near misses involving therapeutic radiation, imaging/simulation, and patient care (not involving radiation), unsafe conditions, operational issues, and accolades/suggestions. Reports were analyzed according to impact on various steps in the process of care. Actions made in response to reports were assessed and characterized by intervention reliability.

RESULTS

A total of 1125 reports were submitted in the first 23 months. For all three departments, therapeutic radiation incidents and near misses consisted of less than one-third of all reports submitted. For the midsized academic department, operational issues and unsafe conditions comprised the largest percentage of reports (70%). Although the majority of reports impacted steps related to the technical aspects of treatment (simulation, planning, and treatment delivery), 20% impacted other steps such as scheduling or clinic visits. More than 160 actions were performed in response to reports. Of these actions, 63 were quality improvement interventions to improve practices, while 97 were learning actions for raising awareness.

CONCLUSIONS

We have developed an ILS that identifies issues related to the entire process of care delivery in radiation oncology, as evidenced by frequent and varied reported events. By identifying a broad spectrum of issues in a department, opportunities for improvement can be identified.

Risk factors for near-miss events and safety incidents in pediatric radiation therapy

BACKGROUND AND PURPOSE

Factors contributing to safety- or quality-related incidents (e.g. variances) in children are unknown. We identified clinical and RT treatment variables associated with risk for variances in a pediatric cohort.

MATERIALS AND METHODS

Using our institution's incident learning system, 81 patients age ≤21 years old who experienced variances were compared to 191 pediatric patients without variances. Clinical and RT treatment variables were evaluated as potential predictors for variances using univariate and multivariate analyses.

RESULTS

Variances were primarily documentation errors (n = 46, 57%) and were most commonly detected during treatment planning (n = 14, 21%). Treatment planning errors constituted the majority (n = 16 out of 29, 55%) of near-misses and safety incidents (NMSI), which excludes workflow incidents. Therapists reported the majority of variances (n = 50, 62%). Physician cross-coverage (OR = 2.1, 95% CI = 1.04-4.38) and 3D conformal RT (OR = 2.3, 95% CI = 1.11-4.69) increased variance risk. Conversely, age >14 years (OR = 0.5, 95% CI = 0.28-0.88) and diagnosis of abdominal tumor (OR = 0.2, 95% CI = 0.04-0.59) decreased variance risk.

CONCLUSIONS

Variances in children occurred in early treatment phases, but were detected at later workflow stages. Quality measures should be implemented during early treatment phases with a focus on younger children and those cared for by cross-covering physicians.

Incident learning in radiation oncology: A review

Incident learning is a key component for maintaining safety and quality in healthcare. Its use is well established and supported by professional society recommendations, regulations and accreditation, and objective evidence. There is an active interest in incident learning systems (ILS) in radiation oncology, with over 40 publications since 2010. This article is intended as a comprehensive topic review of ILS in radiation oncology, including history and summary of existing literature, nomenclature and categorization schemas, operational aspects of ILS at the institutional level including event handling and root cause analysis, and national and international ILS for shared learning. Core principles of patient safety in the context of ILS are discussed, including the systems view of error, culture of safety, and contributing factors such as cognitive bias. Finally, the topics of medical error disclosure and second victim syndrome are discussed. In spite of the rapid progress and understanding of ILS, challenges remain in applying ILS to the radiation oncology context. This comprehensive review may serve as a springboard for further work.

Risk factors for radiotherapy incidents: a single institutional experience

We aimed to analyze risk factors for incidents occurring during the practice of external beam radiotherapy (EBRT) at a single Japanese center. Treatment data for EBRT from June 2014 to March 2017 were collected. Data from incident reports submitted during this period were reviewed. Near-miss cases were not included. Risk factors for incidents, including patient characteristics and treatment-related factors, were explored using uni- and multivariate analyses. Factors contributing to each incident were also retrospectively categorized according to the recommendations of the American Association of Physicists in Medicine (AAPM). A total of 2887 patients were treated during the study period, and 26 incidents occurred (0.90% per patient). Previous history of radiotherapy and large fraction size were identified as risk factors for incidents by univariate analysis. Only previous history of radiotherapy was detected as a risk factor in multivariate analysis. Identified categories of contributing factors were human behavior (50.0%), communication (40.6%), and technical (9.4%). The incident rate of EBRT was 0.90% per patient in our institution. Previous history of radiotherapy and large fraction size were detected as risk factors for incidents. Human behavior and communication errors were identified as contributing factors for most incidents.

A patient safety education program in a medical physics residency

Education in patient safety and quality of care is a requirement for radiation oncology residency programs according to accrediting agencies. However, recent surveys indicate that most programs lack a formal program to support this learning. The aim of this report was to address this gap and share experiences with a structured educational program on quality and safety designed specifically for medical physics therapy residencies. Five key topic areas were identified, drawn from published recommendations on safety and quality. A didactic component was developed, which includes an extensive reading list supported by a series of lectures. This was coupled with practice-based learning which includes one project, for example, failure modes and effect analysis exercise, and also continued participation in the departmental incident learning system including a root-cause analysis exercise. Performance was evaluated through quizzes, presentations, and reports. Over the period of 2014-2016, five medical physics residents successfully completed the program. Evaluations indicated that the residents had a positive experience. In addition to educating physics residents this program may be adapted for medical physics graduate programs or certificate programs, radiation oncology residencies, or as a self-directed educational project for practicing physicists. Future directions might include a system that coordinates between medical training centers such as a resident exchange program.

A nationwide investigation of radiation therapy event reporting-and-learning systems: Can standards be improved?

INTRODUCTION

Variation exists between event reporting-and-learning systems utilised in radiation therapy. Due to the impact of errors associated with this field of medicine, evidence-based and rigorous systems are imperative. The implementation of such systems facilitates the reactive enhancement of patient safety following an event. The purpose of this study was to evaluate Irish event reporting-and-learning procedures against the current literature using a developed evidence-based process map, and to propose recommendations as to how the national standard could be improved.

METHODS

Radiation Therapy Service Managers of all Irish radiation therapy institutions (n = 12) were invited to participate in an anonymous online questionnaire. Included in the questionnaire was a reporting-and-learning process map developed from evidence-based literature, which was used to assess the institution's practice through the use of vignettes. Frequency analysis of closed-ended questions and thematic analysis of open-ended questions was performed to assess the data.

RESULTS

A 91.7% response rate was achieved. The following areas were found to have the most variation with the evidence-based process map: event classification, external reporting, and dissemination of lessons-learned to a wider audience. Recommendations to standardise practice were made.

CONCLUSION

Opportunities for improvement exist within event reporting-and-learning systems of Irish radiation therapy institutions and recommendations have been made on these. These findings can provide learning for other countries with similar reporting systems.

Improving the quality of radiation oncology: 10years' experience of QUATRO audits in the IAEA Europe Region

BACKGROUND AND PURPOSE

The IAEA has developed a methodology for comprehensive quality audits of radiotherapy practices called Quality Assurance Team for Radiation Oncology (QUATRO). This study explores the factors that impacted quality of care among QUATRO audited centres in the IAEA Europe Region.

MATERIALS AND METHODS

The 31 QUATRO reports collected over 10years include extensive data describing the quality of radiotherapy at the audited centres. A coding key was developed to aggregate and review these data in terms of recommendations for improvement and positive findings (commendations).

RESULTS

Overall 759 recommendations and 600 commendations were given. Eight centres recognized as centres of competence differed from other centres mostly because they operated complete quality management systems and were adequately staffed. Other centres had excessive staff workloads and many gaps in the process of care. Insufficient equipment levels were prevalent. Patient centredness, communication, dosimetry, quality control and radiation protection were frequently commended by QUATRO.

CONCLUSIONS

This analysis points to barriers to quality care such as insufficient staffing, education/training, equipment and lack of quality management. It highlights the correlation between the human resources availability and quality of care. It has also identified common action items for enhancing quality of radiotherapy programmes in the Region.

Are we making an impact with incident learning systems? Analysis of quality improvement interventions using total body irradiation as a model system

PURPOSE

Despite increasing interest in incident learning systems (ILS) to improve safety and quality in radiation oncology, little is known about interventions developed in response to safety data. We used total body irradiation (TBI) as a model system to study the effectiveness of interventions from our institutional ILS.

METHODS AND MATERIALS

Near-miss event reports specific to TBI were identified from a departmental ILS from March 2012 to December 2015. The near-miss risk index was rated at multidisciplinary review from 0 (no potential harm) to 4 (critical potential harm). Interventions were analyzed for effectiveness with a schema adapted from The Joint Commission and other agencies: "most reliable" (eg, forcing functions, automation), "somewhat reliable" (eg, checklists, standardization), and "least reliable" (eg, training, rules, procedures). Causal factors of each event were drawn from the casual factor schema used in radiation oncology ILS.

RESULTS

Of 4007 safety-related reports, 266 reports pertained to TBI. TBI reports had a somewhat higher proportion of high-risk events (near-miss risk index 3-4) compared with non-TBI reports (25% vs 17%, P = .0045). A total of 117 interventions were implemented. The reliability indicators for the interventions were: most reliable (11% of interventions), somewhat reliable (17%), and least reliable (72%). Interventions were more likely to be applied to high-risk events (54% vs 41%, P = .03). There was a pattern of high-reliability interventions with increased risk score of events. Events involving human error (eg, slips) and equipment/information technology lent themselves more often to high-reliability interventions. Events related to communication, standardization, and training were associated with low-reliability interventions.

CONCLUSIONS

Over a 3.5-year period, 117 quality improvement strategies were developed for TBI based on ILS. Interventions were more likely to be applied to high-risk events and high-risk events were more likely to be associated with high-quality interventions. These results may be useful to institutions seeking to develop interventions based on ILS data.

Safety Considerations in Stereotactic Body Radiation Therapy

Although many error pathways are common to both stereotactic body radiation therapy (SBRT) and conventional radiation therapy, SBRT presents a special set of challenges including short treatment courses and high-doses, an enhanced reliance on imaging, technical challenges associated with commissioning, special resource requirements for staff and training, and workflow differences. Emerging data also suggest that errors occur at a higher rate in SBRT treatments. Furthermore, when errors do occur they often have a greater effect on SBRT treatments. Given these challenges, it is important to understand and employ systematic approaches to ensure the quality and safety of SBRT treatment. Here, we outline the pathways by which error can occur in SBRT, illustrated through a series of case studies, and highlight 9 specific well-established tools to either reduce error or minimize its effect to the patient or both.

Improving patient safety and workflow efficiency with standardized pretreatment radiation therapist chart reviews

PURPOSE

Radiation therapists play a critical role in ensuring patient safety; however, they are sometimes given insufficient time to perform quality assurance (QA) of a patient's treatment chart and documentation before the start of treatment. In this work, we show the benefits of introducing a formal therapist prestart QA checklist, completed in a quiet space well in advance of treatment, into our workflow.

METHODS AND MATERIALS

A therapist prestart QA checklist was created by analyzing in-house variance reports and treatment unit delays over 6 months. Therapists were then given dedicated time and workspace to perform their checks within the dosimetry office of our department. The effectiveness of the checklist was quantified by recording the percentage of charts that underwent QA before treatment, the percentage of charts with errors needing intervention, and treatment unit delays during a nearly 2-year period. The frequency and types of errors found by the prestart QA were also recorded.

RESULTS

Through the use of therapist prestart QA, instances of treatment unit delays were reduced by up to a factor of 9 during the first year of the program. At the outset of this new initiative, nearly 40% of charts had errors requiring intervention, with the majority being scheduling related. With upstream workflow changes and automation, this was reduced over the period of a year to about 10%.

CONCLUSIONS

The number of treatment unit delays was dramatically reduced by using a formal therapist prestart QA checklist completed well in advance of treatment. The data collected via the checklist continue to be used for further quality improvement efforts.

A Review of Healthcare Failure Mode and Effects Analysis (HFMEA) in Radiotherapy

This paper presents a review of risk analyses in radiotherapy (RT) processes carried out by using Healthcare Failure Mode Effect Analysis (HFMEA) methodology, a qualitative method that proactively identifies risks to patients and corrects medical errors before they occur. This literature review was performed to provide an overview of how to approach the development of HFMEA applications in modern RT procedures, comparing recently published research conducted to support proactive programs to identify risks. On the basis of the reviewed literature, the paper suggests HFMEA shortcomings that need to be addressed.

The report of Task Group 100 of the AAPM: Application of risk analysis methods to radiation therapy quality management

The increasing complexity of modern radiation therapy planning and delivery challenges traditional prescriptive quality management (QM) methods, such as many of those included in guidelines published by organizations such as the AAPM, ASTRO, ACR, ESTRO, and IAEA. These prescriptive guidelines have traditionally focused on monitoring all aspects of the functional performance of radiotherapy (RT) equipment by comparing parameters against tolerances set at strict but achievable values. Many errors that occur in radiation oncology are not due to failures in devices and software; rather they are failures in workflow and process. A systematic understanding of the likelihood and clinical impact of possible failures throughout a course of radiotherapy is needed to direct limit QM resources efficiently to produce maximum safety and quality of patient care. Task Group 100 of the AAPM has taken a broad view of these issues and has developed a framework for designing QM activities, based on estimates of the probability of identified failures and their clinical outcome through the RT planning and delivery process. The Task Group has chosen a specific radiotherapy process required for "intensity modulated radiation therapy (IMRT)" as a case study. The goal of this work is to apply modern risk-based analysis techniques to this complex RT process in order to demonstrate to the RT community that such techniques may help identify more effective and efficient ways to enhance the safety and quality of our treatment processes. The task group generated by consensus an example quality management program strategy for the IMRT process performed at the institution of one of the authors. This report describes the methodology and nomenclature developed, presents the process maps, FMEAs, fault trees, and QM programs developed, and makes suggestions on how this information could be used in the clinic. The development and implementation of risk-assessment techniques will make radiation therapy safer and more efficient.

International recommendations for national patient safety incident reporting systems: an expert Delphi consensus-building process

BACKGROUND

Patient safety incident reporting systems (PSRS) have been established for over a decade, but uncertainty remains regarding the role that they can and ought to play in quantifying healthcare-related harm and improving care.

OBJECTIVE

To establish international, expert consensus on the purpose of PSRS regarding monitoring and learning from incidents and developing recommendations for their future role.

METHODS

After a scoping review of the literature, semi-structured interviews with experts in PSRS were conducted. Based on these findings, a survey-based questionnaire was developed and subsequently completed by a larger expert panel. Using a Delphi approach, consensus was reached regarding the ideal role of PSRSs. Recommendations for best practice were devised.

RESULTS

Forty recommendations emerged from the Delphi procedure on the role and use of PSRS. Experts agreed reporting system should not be used as an epidemiological tool to monitor the rate of harm over time or to appraise the relative safety of hospitals. They agreed reporting is a valuable mechanism for identifying organisational safety needs. The benefit of a national system was clear with respect to medication error, device failures, hospital-acquired infections and never events as these problems often require solutions at a national level. Experts recommended training for senior healthcare professionals in incident investigation. Consensus recommendation was for hospitals to take responsibility for creating safety solutions locally that could be shared nationally.

CONCLUSIONS

We obtained reasonable consensus among experts on aims and specifications of PSRS. This information can be used to reflect on existing and future PSRS, and their role within the wider patient safety landscape. The role of PSRS as instruments for learning needs to be elaborated and developed further internationally.

Ten-year trends in safe radiation therapy delivery and results of a radiation therapy quality assurance intervention

PURPOSE

This study reviews our institutional error data and assesses the effectiveness of a policy implemented January 1, 2011, as a "no rushed treatment" initiative to avoid universal, large-scale replanning for all patients in the event that a treatment unit is down for ≤1 day.

METHODS AND MATERIALS

Radiation error data between January 1, 2004, and December 31, 2014, were reviewed to determine absolute delivery error rates. Variables were compared (using a χ(2) or Fisher exact test) before and after the policy change, including planning versus delivery error status and differences in error type. We also evaluated time of day in relation to therapist shift change, deviation from scheduled time, and weekend treatment as predictors of error using a test of proportions or χ(2) test.

RESULTS

Treatment delivery error rate over the entire period was 0.18% per fraction; the rate before intervention was 0.24% and after was 0.08%, P < .001. For the 5 years for which detailed records were available (2010-2014), 109 delivery errors were reported. Delivery error rate was 0.09%; before intervention 0.15% versus after, 0.08% (P = .005) and 94% were level 1 errors. Fifty-six percent were primary planning errors and 44% were primary delivery errors. Before intervention, large-scale replanning occurred 18 times/year versus 4.5/year after, with 21% versus 12% of errors directly attributable to large-scale replanning. Fourteen error reports specifically implicated a rushed environment as causal. There was no significant difference in error rate based on time of day (P = .631). Error rates were higher for weekend simulation and treatments, 1.3% versus 0.09% per fraction (P < .001).

CONCLUSIONS

Delivery error rates at our institution were similar compared with published series from other academic institutions. A significant improvement in delivery error rate was appreciated after implementation of a "no rushed treatment" initiative. A significantly higher error rate for weekend treatments was noted, warranting consideration of additional quality assurance measures.

Incident learning in pursuit of high reliability: implementing a comprehensive, low-threshold reporting program in a large, multisite radiation oncology department

BACKGROUND

Incident learning programs have been recognized as cornerstones of safety and quality assurance in so-called high reliability organizations in industries such as aviation and nuclear power. High reliability organizations are distinguished by their drive to continuously identify and proactively address a broad spectrum of latent safety issues. Many radiation oncology institutions have reported on their experience in tracking and analyzing adverse events and near misses but few have incorporated the principles of high reliability into their programs. Most programs have focused on the reporting and retrospective analysis of a relatively small number of significant adverse events and near misses. To advance a large, multisite radiation oncology department toward high reliability, a comprehensive, cost-effective, electronic condition reporting program was launched to enable the identification of a broad spectrum of latent system failures, which would then be addressed through a continuous quality improvement process.

METHODS

A comprehensive program, including policies, work flows, and information system, was designed and implemented, with use of a low reporting threshold to focus on precursors to adverse events.

RESULTS

In a 46-month period from March 2011 through December 2014, a total of 8,504 conditions (average, 185 per month, 1 per patient treated, 3.9 per 100 fractions [individual treatments]) were reported. Some 77.9% of clinical staff members reported at least 1 condition. Ninety-eight percent of conditions were classified in the lowest two of four severity levels, providing the opportunity to address conditions before they contribute to adverse events.

CONCLUSIONS

Results after approximately four years show excellent employee engagement, a sustained rate of reporting, and a focus on low-level issues leading to proactive quality improvement interventions.