Analysis of human errors in the operation of various treatment planning systems over a 10-year period

The present study aimed to summarize and report data on errors related to treatment planning, which were collected by medical physicists. The following analyses were performed based on the 10-year error report data: (1) listing of high-risk errors that occurred and (2) the relationship between the number of treatments and error rates, (3) usefulness of the Automated Plan Checking System (APCS) with the Eclipse Scripting Application Programming Interface and (4) the relationship between human factors and error rates. Differences in error rates were observed before and after the use of APCS. APCS reduced the error rate by ~1% for high-risk errors and 3% for low-risk errors. The number of treatments was negatively correlated with error rates. Therefore, we examined the relationship between the workload of medical physicists and error occurrence and revealed that a very large workload may contribute to overlooking errors. Meanwhile, an increase in the number of medical physicists may lead to the detection of more errors. The number of errors was correlated with the number of physicians with less clinical experience; the error rates were higher when there were more physicians with less experience. This is likely due to the lack of training among clinically inexperienced physicians. An environment to provide adequate training is important, as inexperience in clinical practice can easily and directly lead to the occurrence of errors. In any environment, the need for additional plan checkers is an essential factor for eliminating errors.

Development and clinical implementation of a digital system for risk assessments for radiation therapy

Before introducing new treatment techniques, an investigation of hazards due to unintentional radiation exposures is a reasonable activity for proactively increasing patient safety. As dedicated software is scarce, we developed a tool for risk assessment to design a quality management program based on best practice methods, i.e., process mapping, failure modes and effects analysis and fault tree analysis. Implemented as a web database application, a single dataset was used to describe the treatment process and its failure modes. The design of the system and dataset allowed failure modes to be represented both visually as fault trees and in a tabular form. Following the commissioning of the software for our department, previously conducted risk assessments were migrated to the new system after being fully re-assessed which revealed a shift in risk priorities. Furthermore, a weighting factor was investigated to bring risk levels of the migrated assessments into perspective. The compensation did not affect high priorities but did re-prioritize in the midrange of the ranking. We conclude that the tool is suitable to conduct multiple risk assessments and concomitantly keep track of the overall quality management activities.

Process failure mode and effects analysis for external beam radiotherapy: Introducing a literature-based template and a novel action priority

PURPOSE

The first aim of the study was to create a general template for analyzing potential failures in external beam radiotherapy, EBRT, using the process failure mode and effects analysis (PFMEA). The second aim was to modify the action priority (AP), a novel prioritization method originally introduced by the Automotive Industry Action Group (AIAG), to work with different severity, occurrence, and detection rating systems used in radiation oncology.

METHODS AND MATERIALS

The AIAG PFMEA approach was employed in combination with an extensive literature survey to develop the EBRT-PFMEA template. Subsets of high-risk failure modes found through the literature survey were added to the template where applicable. Our modified AP for radiation oncology (RO AP) was defined using a weighted sum of severity, occurrence, and detectability. Then, Monte Carlo simulations were conducted to compare the original AIAG AP, the RO AP, and the risk priority number (RPN). The results of the simulations were used to determine the number of additional corrective actions per failure mode and to parametrize the RO AP to our department's rating system.

RESULTS

An EBRT-PFMEA template comprising 75 high-risk failure modes could be compiled. The AIAG AP required 1.7 additional corrective actions per failure mode, while the RO AP ranged from 1.3 to 3.5, and the RPN required 3.6. The RO AP could be parametrized so that it suited our rating system and evaluated severity, occurrence, and detection ratings equally to the AIAG AP.

CONCLUSIONS

An adjustable EBRT-PFMEA template is provided which can be used as a practical starting point for creating institution-specific templates. Moreover, the RO AP introduces transparent action levels that can be adapted to any rating system.

The report of AAPM task group 288: Recommendations for guiding radiotherapy event narratives

Incident reporting and learning systems provide an opportunity to identify systemic vulnerabilities that contribute to incidents and potentially degrade quality. The narrative of an incident is intended to provide a clear, easy to understand description of an incident. Unclear, incomplete or poorly organized narratives compromise the ability to learn from them. This report provides guidance for drafting effective narratives, with particular attention to the use of narratives in incident reporting and learning systems (IRLS). Examples are given that compare effective and less than effective narratives. This report is mostly directed to organizations that maintain IRLS, but also may be helpful for individuals who desire to write a useful narrative for entry into such a system. Recommendations include the following: (1) Systems should allow a one- or two-sentence, free-text synopsis of an incident without guessing at causes; (2) Information included should form a sequence of events with chronology; and (3) Reporting and learning systems should consider using the headings suggested to guide the reporter through the narrative: (a) incident occurrences and actions by role; (b) prior circumstances and actions; (c) method by which the incident was identified; (d) equipment related details if relevant; (e) recovery actions by role; (f) relevant time span between responses; (g) and how individuals affected during or immediately after incident. When possible and appropriate, supplementary information including relevant data elements should be included using numerical scales or drop-down choices outside of the narrative. Information that should not be included in the narrative includes: (a) patient health information (PHI); (b) conjecture or blame; (c) jargon abbreviations or details without specifying their significance; (d) causal analysis.

Causal analysis of radiotherapy safety incidents based on a hybrid model of HFACS and Bayesian network

Objective

This research investigates the role of human factors of all hierarchical levels in radiotherapy safety incidents and examines their interconnections.

Methods

Utilizing the human factor analysis and classification system (HFACS) and Bayesian network (BN) methodologies, we created a BN-HFACS model to comprehensively analyze human factors, integrating the hierarchical structure. We examined 81 radiotherapy incidents from the radiation oncology incident learning system (RO-ILS), conducting a qualitative analysis using HFACS. Subsequently, parametric learning was applied to the derived data, and the prior probabilities of human factors were calculated at each BN-HFACS model level. Finally, a sensitivity analysis was conducted to identify the human factors with the greatest influence on unsafe acts.

Results

The majority of safety incidents reported on RO-ILS were traced back to the treatment planning phase, with skill errors and habitual violations being the primary unsafe acts causing these incidents. The sensitivity analysis highlighted that the condition of the operators, personnel factors, and environmental factors significantly influenced the occurrence of incidents. Additionally, it underscored the importance of organizational climate and organizational process in triggering unsafe acts.

Conclusion

Our findings suggest a strong association between upper-level human factors and unsafe acts among radiotherapy incidents in RO-ILS. To enhance radiation therapy safety and reduce incidents, interventions targeting these key factors are recommended.

Improving Quality Assurance in a Radiation Oncology Using ARIA Visual Care Path

PURPOSE

Errors and incidents may occur at any point within radiotherapy (RT). The aim of the present retrospective analysis is to evaluate the impact of a customized ARIA Visual Care Path (VCP) on quality assurance (QA) for the RT process.

MATERIALS AND METHODS

The ARIA VCP was implemented in June 2019. The following tasks were customized and independently verified (by independent checks from radiation oncologists, medical physics, and radiation therapists): simulation, treatment planning, treatment start verification, and treatment completion. A retrospective analysis of 105 random and unselected patients was performed, and 945 tasks were reviewed. Patients' reports were categorized based on treatment years period: 2019-2020 (A); 2021 (B); and 2022-2023 (C). The QA metrics included data for timeliness of task completion and data for minor and major incidents. The major incidents were defined as incorrect prescriptions of RT dose, the use of different immobilization systems during RT compared to the simulation, the absence of surface-guided RT data for patients' positioning, incorrect dosimetric QA for treatment plans, and failure to complete RT as originally planned. A sample size of approximately 100 was able to obtain an upper limit of 95% confidence interval below 5-10% in the case of zero or one major incident.

RESULTS

From June 2019 to December 2023, 5300 patients were treated in our RT department, an average of 1300 patients per year. For the purpose of this analysis, one hundred and five patients were chosen for the study and were subsequently evaluated. All RT staff achieved a 100% compliance rate in the ARIA VCP timely completion. A total of 36 patients were treated in Period A, 34 in Period B, and 35 in Period C. No major incidents were identified, demonstrating a major incident rate of 0.0% (95% CI 0.0-3.5%). A total of 26 out of 945 analyzed tasks (3.8%) were reported as minor incidents: absence of positioning photo in 32 cases, lack of patients' photo, and absence of plan documents in 4 cases. When comparing periods, incidents were statistically less frequent in Period C.

CONCLUSIONS

Although the present analysis has some limitations, its outcomes demonstrated that software for the RT workflow, which is fully integrated with both the record-and-verify and treatment planning systems, can effectively manage the patient's care path. Implementing the ARIA VCP improved the efficiency of the RT care path workflow, reducing the risk of major and minor incidents.

Unified risk analysis in radiation therapy

PURPOSE

The increasing complexity of new treatment methods as well as the Information Technology (IT) infrastructure within radiotherapy require new methods for risk analysis. This work presents a methodology on how to model the treatment process of radiotherapy in different levels. This subdivision makes it possible to perform workflow-specific risk analysis and to assess the impact of IT risks on the overall treatment workflow.

METHODS

A Unified Modeling Language (UML) activity diagram is used to model the workflows. The subdivision of the workflows into different levels is done with the help of swim lanes. The model created in this way is exported in an xml-compatible format and stored in a database with the help of a Python program.

RESULTS

Based on an existing risk analysis, the workflows CT Appointment, Glioblastoma Multiforme, and Deep Inspiration Breath Hold (DIBH) were modeled in detail. Part of the analysis are automatically generated workflow-specific risk matrices including risks of medical devices incorporated into a specific workflow. In addition, SQL queries allow to quickly retrieve e.g., the details of the medical device network installed in a department.

CONCLUSION

Activity diagrams of UML can be used to model workflows in radiotherapy. Through this, a connection between the different levels of the entire workflow can be established and workflow-specific risk analysis is possible.

Implementation of a knowledge-based decision support system for treatment plan auditing through automation

BACKGROUND

Independent auditing is a necessary component of a comprehensive quality assurance (QA) program and can also be utilized for continuous quality improvement (QI) in various radiotherapy processes. Two senior physicists at our institution have been performing a time intensive manual audit of cross-campus treatment plans annually, with the aim of further standardizing our planning procedures, updating policies and guidelines, and providing training opportunities of all staff members.

PURPOSE

A knowledge-based automated anomaly-detection algorithm to provide decision support and strengthen our manual retrospective plan auditing process was developed. This standardized and improved the efficiency of the assessment of our external beam radiotherapy (EBRT) treatment planning across all eight campuses of our institution.

METHODS

A total of 843 external beam radiotherapy plans for 721 lung patients from January 2020 to March 2021 were automatically acquired from our clinical treatment planning and management systems. From each plan, 44 parameters were automatically extracted and pre-processed. A knowledge-based anomaly detection algorithm, namely, "isolation forest" (iForest), was then applied to the plan dataset. An anomaly score was determined for each plan using recursive partitioning mechanism. Top 20 plans ranked with the highest anomaly scores for each treatment technique (2D/3D/IMRT/VMAT/SBRT) including auto-populated parameters were used to guide the manual auditing process and validated by two plan auditors.

RESULTS

The two auditors verified that 75.6% plans with the highest iForest anomaly scores have similar concerning qualities that may lead to actionable recommendations for our planning procedures and staff training materials. The time to audit a chart was approximately 20.8 min on average when done manually and 14.0 min when done with the iForest guidance. Approximately 6.8 min were saved per chart with the iForest method. For our typical internal audit review of 250 charts annually, the total time savings are approximately 30 hr per year.

CONCLUSION

iForest effectively detects anomalous plans and strengthens our cross-campus manual plan auditing procedure by adding decision support and further improve standardization. Due to the use of automation, this method was efficient and will be used to establish a standard plan auditing procedure, which could occur more frequently.

Safety in radiation oncology (SAFRON): Learning about incident causes and safety barriers in external beam radiotherapy

PURPOSE

Safety in Radiation Oncology (SAFRON) is a reporting and learning system on radiotherapy and radionuclide therapy incidents and near misses. The primary aim of this paper is to examine whether any discernible patterns exist in the causes of reported incidents and safety barriers within the SAFRON system concerning external beam radiotherapy.

METHODS AND MATERIALS

This study focuses on external beam radiotherapy incidents, reviewing 1685 reports since the inception of SAFRON until December 2021. Reports that did not identify causes of incidents and safety barriers were excluded from the final study population.

RESULTS

Simple two-dimensional radiotherapy or electron beam therapy were represented by 97 reports, three-dimensional conformal radiotherapy by 39 reports, modulated arc therapy by 12 reports, intensity modulated radiation therapy by 11 reports, stereotactic radiosurgery by 4 reports, and radiotherapy with protons or other particles by 1 report, while for 92 of them, no information on treatment method had been provided. Most of the reported incidents were minor incidents and were discovered by the radiation therapist. Inadequate direction/information in staff communication was the most frequently reported cause of incident, and regular independent chart check was the most common safety barrier.

CONCLUSIONS

The results indicate that the majority of incidents were reported by radiation therapists, and the majority of these incidents were classified as minor. Communication problems and failure to follow standards/procedures/practices were the most frequent causes of incidents. Furthermore, regular independent chart checking was the most frequently identified safety barrier.

Risk management patterns in radiation oncology-results of a national survey within the framework of the Patient Safety in German Radiation Oncology (PaSaGeRO) project

PURPOSE

Risk management (RM) is a key component of patient safety in radiation oncology (RO). We investigated current approaches on RM in German RO within the framework of the Patient Safety in German Radiation Oncology (PaSaGeRO) project. Aim was not only to evaluate a status quo of RM purposes but furthermore to discover challenges for sustainable RM that should be addressed in future research and recommendations.

METHODS

An online survey was conducted from June to August 2021, consisting of 18 items on prospective and reactive RM, protagonists of RM, and self-assessment concerning RM. The survey was designed using LimeSurvey and invitations were sent by e‑mail. Answers were requested once per institution.

RESULTS

In all, 48 completed questionnaires from university hospitals, general and non-academic hospitals, and private practices were received and considered for evaluation. Prospective and reactive RM was commonly conducted within interprofessional teams; 88% of all institutions performed prospective risk analyses. Most institutions (71%) reported incidents or near-events using multiple reporting systems. Results were presented to the team in 71% for prospective analyses and 85% for analyses of incidents. Risk conferences take place in 46% of institutions. 42% nominated a manager/committee for RM. Knowledge concerning RM was mostly rated "satisfying" (44%). However, 65% of all institutions require more information about RM by professional societies.

CONCLUSION

Our results revealed heterogeneous patterns of RM in RO departments, although most departments adhered to common recommendations. Identified mismatches between recommendations and implementation of RM provide baseline data for future research and support definition of teaching content.

[Comparison of incident reporting and learning systems for radiation oncology in France and abroad]

Reporting and learning are key components of quality and safety in radiotherapy. Each event must be reported to national authorities if considered significant according to national criteria. Lessons learnt from analysis of causal factors are primordial to decrease the risk of reoccurrence or the severity of further events. Thanks to national or international, mandatory or voluntary incidents reporting systems, and experience feedbacks, various sources of learning are available to improve risk management. This article aims to compare the regulations about mandatory declarations of significant events and describe national or international incident reporting and learning systems available.

Multi-Institutional Stereotactic Body Radiation Therapy Incident Learning: Evaluation of Safety Barriers Using a Human Factors Analysis and Classification System

OBJECTIVES

Stereotactic body radiation therapy (SBRT) can improve therapeutic ratios and patient convenience, but delivering higher doses per fraction increases the potential for patient harm. Incident learning systems (ILSs) are being increasingly adopted in radiation oncology to analyze reported events. This study used an ILS coupled with a Human Factor Analysis and Classification System (HFACS) and barriers management to investigate the origin and detection of SBRT events and to elucidate how safeguards can fail allowing errors to propagate through the treatment process.

METHODS

Reported SBRT events were reviewed using an in-house ILS at 4 institutions over 2014-2019. Each institution used a customized care path describing their SBRT processes, including designated safeguards to prevent error propagation. Incidents were assigned a severity score based on the American Association of Physicists in Medicine Task Group Report 275. An HFACS system analyzed failing safeguards.

RESULTS

One hundred sixty events were analyzed with 106 near misses (66.2%) and 54 incidents (33.8%). Fifty incidents were designated as low severity, with 4 considered medium severity. Incidents most often originated in the treatment planning stage (38.1%) and were caught during the pretreatment review and verification stage (37.5%) and treatment delivery stage (31.2%). An HFACS revealed that safeguard failures were attributed to human error (95.2%), routine violation (4.2%), and exceptional violation (0.5%) and driven by personnel factors 32.1% of the time, and operator condition also 32.1% of the time.

CONCLUSIONS

Improving communication and documentation, reducing time pressures, distractions, and high workload should guide proposed improvements to safeguards in radiation oncology.

Errors detected during physics plan review for external beam radiotherapy

Background and purpose

Risk management in radiotherapy is of high importance. There is not much data published on errors occurring in the treatment planning process of external beam techniques. The aim of this study was to investigate errors occurring during physics plan review in external beam radiotherapy.

Materials and methods

Over a period of 14 months errors observed during the physical review process are reported. The errors were grouped and evaluated regarding treatment machine, technique, and treatment site. In addition, a correlation between frequency of errors and staff shortage was analyzed.

Results

Subgroups of grave errors (g-errors) and slight errors (s-errors) were defined to consider the different impact on the patient and clinical workflow of the errors. In 1056 plans reviewed, 110 errors (41 g-errors, 69 s-errors) were detected. The most common g-errors and s-errors were "Wrong gantry angle at setup field" (n = 19) and "Wrong field label" (n = 24), respectively. A correlation of number of errors and treatment machine, technique, or anatomical site could not be found. No correlation between staff shortage and number of errors was observed.

Conclusions

The process of reviewing treatment plans is a relevant topic to consider in risk analysis of the radiotherapy workflow. The review process could be improved by enhancements in the treatment planning systems, use of digital dose prescription, and treatment planning templates.

The impact of COVID-19 workflow changes on radiation oncology incident reporting

Background

The Ottawa Hospital's Radiation Oncology program maintains the Incident Learning System (ILS)—a quality assurance program that consists of report submissions of errors and near misses arising from all major domains of radiation. In March 2020, the department adopted workflow changes to optimize patient and provider safety during the COVID‐19 pandemic.

Purpose

In this study, we analyzed the number and type of ILS submissions pre‐ and postpandemic precautions to assess the impact of COVID‐19‐related workflow changes.

Methods

ILS data was collected over six one‐year time periods between March 2016 and March 2021. For all time periods, the number of ILS submissions were counted. Each ILS submission was analyzed for the specific treatment domain from which it arose and its root cause, explaining the impetus for the error or near miss.

Results

Since the onset of COVID‐19‐related workflow changes, the total number of ILS submissions have reduced by approximately 25%. Similarly, there were 30% fewer ILS submissions per number of treatment courses compared to prepandemic data. There was also an increase in the proportion of “treatment planning” ILS submissions and a 50% reduction in the proportion of “decision to treat” ILS submissions compared to previous years. Root cause analysis revealed there were more incidents attributable to “poor, incomplete, or unclear documentation” during the pandemic year.

Conclusions

COVID‐19 workflow changes were associated with fewer ILS submissions, but a relative increase in submissions stemming from poor documentation and communication. It is imperative to analyze ILS submission data, particularly in a changing work environment, as it highlights the potential and realized mistakes that impact patient and staff safety.

Quality and Safety With Technological Advancements in Radiotherapy: An Overview and Journey Narrative From a Low- and Middle-Income Country Institution

To present an overview of quality and safety in radiotherapy from the context of low- and middle-income countries on the basis of a recently conducted annual meeting of our institution and our experience of implementing an error management system at our center.

Quality and safety improvement with evolving technology in LMIC, a journey described.

Evaluating incident learning systems and safety culture in two radiation oncology departments

INTRODUCTION

Radiation oncology patient pathways are complex. This complexity creates risk and potential for error to occur. Comprehensive safety and quality management programmes have been developed alongside the use of incident learning systems (ILSs) to mitigate risks and errors reaching patients. Robust ILSs rely on the safety culture (SC) within a department. The aim of this study was to assess perceptions and understanding of SC and ILSs in two closely linked radiation oncology departments and to use the results to consider possible quality improvement (QI) of department ILSs and SC.

METHODS

A survey to assess perceptions of SC and the currently used ILSs was distributed to radiation oncologists, radiation therapists and radiation oncology medical physicists in the two departments. The responses of 95 staff were evaluated (63% of staff). The findings were used to determine any areas for improvement in SC and local ILSs.

RESULTS

Differences were shown between the professional cohorts. Barriers to current ILS use were indicated by 67% of respondents. Positive SC was shown in each area assessed: 69% indicated the departments practised a no-blame culture. Barriers identified in one department prompted a QI project to develop a new reporting system and process, improve departmental learning and modify the overall ILS.

CONCLUSION

An understanding of SC and attitudes to ILSs has been established and used to improve ILS reporting, feedback on incidents, departmental learning and the QA program. This can be used for future comparisons as the systems develop.

Safety culture and incident learning systems in radiation oncology: Staff perceptions across Australia and New Zealand

INTRODUCTION

Radiation therapy has a highly complex pathway and uses detailed quality assurance protocols and incident learning systems (ILSs) to mitigate risk; however, errors can still occur. The safety culture (SC) in a department influences its commitment and effectiveness in maintaining patient safety.

METHODS

Perceptions of SC and knowledge and understanding of ILSs and their use were evaluated for radiation oncology staff across Australia and New Zealand (ANZ). A validated healthcare survey tool (the Hospital Survey on Patient Safety Culture) was used, with additional specialty-focussed supporting questions. A total of 220 radiation oncologists, radiation therapists and radiation oncology medical physicists participated.

RESULTS

An overall positive SC was indicated, with strength in teamwork (83.7%), supervisor/manager/leader support (83.3%) and reporting events (77.1%). The weakest areas related to communication about error (63.9%), hospital-level management support (60.5%) and handovers and information exchange (58.0%). Barriers to ILS use included 'it takes too long' and that many respondents must use multiple reporting systems, including mandatory hospital-level systems. These are generally not optimal for specific radiation oncology needs. Varied understanding was indicated of what and when to report.

CONCLUSION

The findings report the ANZ perspective on ILS and SC, highlighting weaknesses, barriers and areas for further investigation. Differences observed in some areas suggest that a unified state, national or bi-national ILS specific to radiation oncology might eliminate multiple reporting systems and reduce reporting time. It could also provide more consistent and robust approaches to incident reporting, information sharing and analysis.

Natural language processing and machine learning to assist radiation oncology incident learning

PURPOSE

To develop a Natural Language Processing (NLP) and Machine Learning (ML) pipeline that can be integrated into an Incident Learning System (ILS) to assist radiation oncology incident learning by semi-automating incident classification. Our goal was to develop ML models that can generate label recommendations, arranged according to their likelihoods, for three data elements in Canadian NSIR-RT taxonomy.

METHODS

Over 6000 incident reports were gathered from the Canadian national ILS as well as our local ILS database. Incident descriptions from these reports were processed using various NLP techniques. The processed data with the expert-generated labels were used to train and evaluate over 500 multi-output ML algorithms. The top three models were identified and tuned for each of three different taxonomy data elements, namely: (1) process step where the incident occurred, (2) problem type of the incident and (3) the contributing factors of the incident. The best-performing model after tuning was identified for each data element and tested on unseen data.

RESULTS

The MultiOutputRegressor extended Linear SVR models performed best on the three data elements. On testing, our models ranked the most appropriate label 1.48 ± 0.03, 1.73 ± 0.05 and 2.66 ± 0.08 for process-step, problem-type and contributing factors respectively.

CONCLUSIONS

We developed NLP-ML models that can perform incident classification. These models will be integrated into our ILS to generate a drop-down menu. This semi-automated feature has the potential to improve the usability, accuracy and efficiency of our radiation oncology ILS.

The role of surface-guided radiation therapy for improving patient safety

Emerging data indicates SGRT could improve safety and quality by preventing errors in its capacity as an independent system in the treatment room. The aim of this work is to investigate the utility of SGRT in the context of safety and quality. Three incident learning systems (ILS) were reviewed to categorize and quantify errors that could have been prevented with SGRT: SAFRON (International Atomic Energy Agency), UW-ILS (University of Washington) and AvIC (Skåne University Hospital). A total of 849/9737 events occurred during the pre-treatment review/verification and treatment stages. Of these, 179 (21%) events were predicted to have been preventable with SGRT. The most common preventable events were wrong isocentre (43%) and incorrect accessories (34%), which appeared at comparable rates among SAFRON and UW-ILS. The proportion of events due to wrong accessories was much smaller in the AvIC ILS, which may be attributable to the mandatory use of SGRT in Sweden. Several case scenarios are presented to demonstrate that SGRT operates as a valuable complement to other quality-improvement tools routinely used in radiotherapy. Cases are noted in which SGRT itself caused incidents. These were mostly related to workflow issues and were of low severity. Severity data indicated that events with the potential to be mitigated by SGRT were of higher severity for all categories except wrong accessories. Improved vendor integration of SGRT systems within the overall workflow could further enhance its clinical utility. SGRT is a valuable tool with the potential to increase patient safety and treatment quality in radiotherapy.

Evaluation of failure modes and effect analysis for routine risk assessment of lung radiotherapy at a UK center

PURPOSE

Explore the feasibility of adopting failure modes and effects analysis (FMEA) for risk assessment of a high volume clinical service at a UK radiotherapy center. Compare hypothetical failure modes to locally reported incidents.

METHOD

An FMEA for a lung radiotherapy service was conducted at a hospital that treats ~ 350 lung cancer patients annually with radical radiotherapy. A multidisciplinary team of seven people was identified including a nominated facilitator. A process map was agreed and failure modes identified and scored independently, final failure modes and scores were then agreed at a face-to-face meeting. Risk stratification methods were explored and staff effort recorded. Radiation incidents related to lung radiotherapy reported locally in a 2-year period were analyzed to determine their relation to the identified failure modes. The final FMEA was therefore a combination of prospective evaluation and retrospective analysis from an incident learning system.

RESULTS

Thirty-six failure modes were identified for the pre-existing clinical service. The top failure modes varied according to the ranking method chosen. The process required 30 h of combined staff time. Over the 2-year period chosen, 38 voluntarily reported incidents were identified as relating to lung radiotherapy. Of these, 13 were not predicted by the identified failure modes, with six relating to delays in the process, three issues with appointment times, one communication error, two instances of a failure to image, and one technical fault deemed unpredictable by the manufacturer. Four additional failure modes were added to the FMEA following the incident analysis.

CONCLUSION

FMEA can be effectively applied to an established high volume service as a risk assessment method. Facilitation by an individual familiar with the FMEA process can reduce resource requirement. Prospective evaluation of risks should be combined with an incident reporting and learning system to produce a more comprehensive analysis of risk.

Improving Incident Reporting in a Hospital-Based Radiation Oncology Department: The Impact of a Customized Crew Resource Training and Event Reporting Intervention

Background

Radiation oncology (RO) is a high-risk environment with an increased potential for error due to the complex automated and manual interactions between heterogeneous teams and advanced technologies. Errors involving procedural deviations­­ can adversely impact patient morbidity and mortality. Under-reporting of errors is common in healthcare for reasons such as fear of retribution, liability, embarrassment, etc. Incident reporting is a proven tool for learning from errors and, when effectively implemented, can improve quality and safety. Crew resource management (CRM) employs just culture principles with a team-based safety system. The pillars of CRM include mandatory error reporting and structured training to proactively identify, learn from, and mitigate incidents. High-reliability organizations, such as commercial aviation, have achieved exemplary safety performance since adopting CRM strategies.

Objective

Our aim was to double the rate of staff error reporting from baseline rates utilizing CRM strategies during a six-month study period in a hospital-based radiation oncology (RO) department.

Methods

This quasi-experimental study involved a retrospective review of reported radiation oncology incidents between January 2015 and March 2016, which helped inform the development and implementation of a two-step custom CRM training and incident learning system (ILS) intervention in May 2016. A convenience sample of approximately 50 RO staff (Staff) performing over 100 external beam and daily brachytherapy treatments participated in weekly training for six months while continuing to report errors on a hospital-enterprise system. A discipline-specific incident learning system (ILS) customized for the department was added during the last three months of the study, enabling staff to identify, characterize, and report incidents and potential errors. Weekly process control charts used to trend incident reporting rates (total number of reported incidents in a given month /1000 fractions), and custom reports characterizing the potential severity as well as the location of incidents along the treatment path, were reviewed, analyzed, and addressed by an RO multidisciplinary project committee established for this study.

Results

A five-fold increase in the monthly reported number of incidents (n = 9.3) was observed during the six-month intervention period as compared to the 16-month pre-intervention period (n = 1.8). A significant increase (>3 sigma) was observed when the custom reporting system was added during the last three study months.

Conclusion

A discipline-specific electronic ILS enabling the characterization of individual RO incidents combined with routine CRM training is an effective method for increasing staff incident reporting and engagement, leading to a more systematic, team-based mitigation process. These combined strategies allowed for real-time reporting, analysis, and learning that can be used to enhance patient safety, improve teamwork, streamline communication, and advance a culture of safety.

Critical success factors for implementation of an incident learning system in radiation oncology department

Aim

The aim of this study was to analyze critical success factors (CSFs) for implementation of an incident learning system (ILS) in a radiation oncology department (ROD) and evaluate the perception of the staff members along this process.

Background

Implementing an ILS is a way to leverage learning from incidents and is a tool for improving patient safety, consisting of a cycle of reporting and analyzing events as well as taking preventive actions. ILS implementation is challenging, requiring specific resources and cultural changes.

Materials and methods

An ILS was designed and implemented based on the CSF identified in the literature review. Before starting the ILS implementation, a structured survey was applied to assess dimensions of patient safety culture. After the period of implementation (7 months), the survey was applied again and compared with the initial assessment, and interviews were performed with staff members to evaluate the overall satisfaction with ILS and CSFs.

Results

Statistically significant improvements were observed in 5 dimensions (12 totals) of the safety culture survey, considering time points before and after the ILS implementation. According to interviewees, "Facilitating committee", "Efficient data collection", "Focus on improvement", "Just culture" and "Feedback to users" were the most relevant CSFs.

Conclusions

The ILS designed and implemented at ROD was perceived as an important tool to support quality and safety initiatives, promoting the improvement in safety culture. The ILS implementation critical success factors were identified and have shown good agreement between the results of the literature and the users' practical perception.

Impact of Simulation-Based Training on Radiation Therapists' Workload, Situation Awareness, and Performance

Purpose

This study aimed to assess the impact of simulation-based training intervention on radiation therapy therapist (RTT) mental workload, situation awareness, and performance during routine quality assurance (QA) and treatment delivery tasks.

Methods and Materials

As part of a prospective institutional review board-approved study, 32 RTTs completed routine QA and treatment delivery tasks on clinical scenarios in a simulation laboratory. Participants, randomized to receive (n = 16) versus not receive (n = 16) simulation-based training had pre- and postintervention assessments of mental workload, situation awareness, and performance. We used linear regression models to compare the postassessment scores between the study groups while controlling for baseline scores. Mental workload was quantified subjectively using the NASA Task Load Index. Situation awareness was quantified subjectively using the situation awareness rating technique and objectively using the situation awareness global assessment technique. Performance was quantified based on procedural compliance (adherence to preset/standard QA timeout tasks) and error detection (detection and correction of embedded treatment planning errors).

Results

Simulation-based training intervention was associated with significant improvements in overall performance (P < .01), but had no significant impact on mental workload or subjective/objective quantifications of situation awareness.

Conclusions

Simulation-based training might be an effective tool to improve RTT performance of QA-related tasks.

Automatic Incident Triage in Radiation Oncology Incident Learning System

The Radiotherapy Incident Reporting and Analysis System (RIRAS) receives incident reports from Radiation Oncology facilities across the US Veterans Health Affairs (VHA) enterprise and Virginia Commonwealth University (VCU). In this work, we propose a computational pipeline for analysis of radiation oncology incident reports. Our pipeline uses machine learning (ML) and natural language processing (NLP) based methods to predict the severity of the incidents reported in the RIRAS platform using the textual description of the reported incidents. These incidents in RIRAS are reviewed by a radiation oncology subject matter expert (SME), who initially triages some incidents based on the salient elements in the incident report. To automate the triage process, we used the data from the VHA treatment centers and the VCU radiation oncology department. We used NLP combined with traditional ML algorithms, including support vector machine (SVM) with linear kernel, and compared it against the transfer learning approach with the universal language model fine-tuning (ULMFiT) algorithm. In RIRAS, severities are divided into four categories; A, B, C, and D, with A being the most severe to D being the least. In this work, we built models to predict High (A & B) vs. Low (C & D) severity instead of all the four categories. Models were evaluated with macro-averaged precision, recall, and F1-Score. The Traditional ML machine learning (SVM-linear) approach did well on the VHA dataset with 0.78 F1-Score but performed poorly on the VCU dataset with 0.5 F1-Score. The transfer learning approach did well on both datasets with 0.81 F1-Score on VHA dataset and 0.68 F1-Score on the VCU dataset. Overall, our methods show promise in automating the triage and severity determination process from radiotherapy incident reports.

The RABBIT risk-based approach to clinical implementation of new technology: SRS as a case study

Radiation oncology technology continues to evolve rapidly, resulting in advanced versions frequently being brought to market. Before a new product is used standard tests are carried out to reduce the risks associated with failure of the equipment to comply with well-established technical specifications. It is much harder to identify and reduce the risks associated with how the new technology is used clinically, such as those related to poor communication and high workload.

To ensure that new technology and techniques are used safely and appropriately the implementation project should be managed by a multidisciplinary team (MDT) made up of representatives from all the relevant professions. The MDT’s role is to agree on the project scope, identify and rank all risks and benefits, and direct resources towards mitigating the highest risks. Before clinical release there should be consensus from the MDT that the benefits of the new technology outweigh the residual risks.

The introduction of initiatives to optimise current practice may involve major changes which can be met with barriers such as limited support from management, insufficient time for MDT meetings, and staff fearful of being shown to have poor practices. To help overcome these challenges our team at St George Hospital Cancer Care Centre has developed a Risk and Benefit Balance Impact Template (RABBIT), which guides an MDT through the rapid implementation and safe use of new technology and techniques with an easy to follow Microsoft Word document.

The implementation of stereotactic radiosurgery is used as a case study to illustrate the RABBIT methodology. The RABBIT is a user-friendly method for a busy radiotherapy clinic to transition to a risk-based MDT approach for the implementation of new technologies and techniques. When staff from all disciplines feel empowered to raise concerns about risks the workplace become inherently safer for patients and staff alike.

Strategies for effective physics plan and chart review in radiation therapy: Report of AAPM Task Group 275

BACKGROUND

While the review of radiotherapy treatment plans and charts by a medical physicist is a key component of safe, high-quality care, very few specific recommendations currently exist for this task.

AIMS

The goal of TG-275 is to provide practical, evidence-based recommendations on physics plan and chart review for radiation therapy. While this report is aimed mainly at medical physicists, others may benefit including dosimetrists, radiation therapists, physicians and other professionals interested in quality management.

METHODS

The scope of the report includes photon/electron external beam radiotherapy (EBRT), proton radiotherapy, as well as high-dose rate (HDR) brachytherapy for gynecological applications (currently the highest volume brachytherapy service in most practices). The following review time points are considered: initial review prior to treatment, weekly review, and end-of-treatment review. The Task Group takes a risk-informed approach to developing recommendations. A failure mode and effects analysis was performed to determine the highest-risk aspects of each process. In the case of photon/electron EBRT, a survey of all American Association of Physicists in Medicine (AAPM) members was also conducted to determine current practices. A draft of this report was provided to the full AAPM membership for comment through a 3-week open-comment period, and the report was revised in response to these comments.

RESULTS

The highest-risk failure modes included 112 failure modes in photon/electron EBRT initial review, 55 in weekly and end-of-treatment review, 24 for initial review specific to proton therapy, and 48 in HDR brachytherapy. A 103-question survey on current practices was released to all AAPM members who self-reported as working in the radiation oncology field. The response rate was 33%. The survey data and risk data were used to inform recommendations.

DISCUSSION

Tables of recommended checks are presented and recommendations for best practice are discussed. Suggestions to software vendors are also provided.

CONCLUSIONS

TG-275 provides specific recommendations for physics plan and chart review which should enhance the safety and quality of care for patients receiving radiation treatments.

Safety First: Developing and Deploying a System to Promote Safety and Quality in Your Clinic

The terms "safety and quality" (SAQ) have become inextricably linked, highly used terms that together encompass a wide range of parameters within medical departments. Safety has always been a priority in radiation oncology; quality assurance has been foundational to our practice. Despite this increased focus and attention on SAQ, the "what" of SAQ remains ill-defined, largely because of the vast number of indicators that fall under this umbrella. Similarly, the "how" of developing and maintaining the highest standards of SAQ is not formulaic and varies based on the unique setting of individual practices. There are several excellent resources available to inform SAQ in radiation oncology, including the American Society for Radiation Oncology's "Safety Is No Accident," which provides an overview of safety and quality standards and resources. This review is intended as a brief summary of key considerations, with the goal of providing a practical framework and context for improving or developing a SAQ program in radiation oncology practices. We believe that the following 10 key elements, drawn from numerous reports that have appeared over the last decade examining this topic, should be considered when conceptualizing a practice-based approach to SAQ: establishing a strong safety culture; establishing a structured program for safety and quality; establishing up-to-date, relevant, and accessible policies and procedures; a system for peer review; systems to assess and reduce risk; an educational program focused on safety and quality; development and review of meaningful quality metrics; utilization of a physics quality control system; well-defined models for staffing, training, and professional development; and finally, validation from external bodies via accreditations and audits. These 10 items are addressed herein.

[Safety in radiationtherapy. Results after 9 years implementation of incidents reporting system]

INTRODUCTION

Radiation therapy (RT) is a complex process that employs high-dose radiation for therapeutic purposes. Incident reporting and analysis, in addition to being a legal requirement in RT, provides information that helps to improve patient safety. This paper describes our experiences over a 9 year period in which a local incident reporting and learning system (SNAI) specific to RT was employed.

MATERIALS AND METHODS

The center has 4 lineal accelerators that treat a total of 1900 patients annually. The first action taken with a view to improving patient safety was the implementation of a multidisciplinary RT safety group (GSRT), who decided to employing a methodology based on incident reporting. For this purpose, a local SNAI was implemented, adapting the ROSEIS incident reporting system used and consolidated by the European Society of Radiation Oncology Therapy (ESTRO). All incidents in which patients received an incorrect RT session were considered adverse events (AE) and were thus analyzed. Finally, the opinion of the professionals involved in relation to the SNAI and the functioning of the safety group was evaluated by means of a survey.

RESULTS

From June 2009 to October 2018, 1708 incidents were recorded, with an increasing incidence observed over time. Approximately 2.5% of the incidents reported were AE. The remainders were events that did not affect the patient. As many as 55% of incidents were detected in the treatment administration phase. Radiotherapy technicians were the professionals who reported more incidents. The majority of recorded cases originated from procedural shortcomings relating to communication or work protocols. Implemented remedial actions were aimed at reducing the frequency of AE and facilitating its early detection. Actions employed were essentially: drafting and revision of protocols and circuits, implementation of checklists, and training actions. Of the workers surveyed, 85% positively valued the incorporation of the SNAI and the existence of a safety group. However, 15% of the professionals considered that the methodology used in the analysis of incidents was not totally objective i.e punitive in nature.

CONCLUSIONS

The safety of the patient receiving RT has been approached from a methodology based on a local SNAI. The analysis of reported incidents has promoted various actions aimed at improving the safety of patients receiving RT. The methodology used has been well received by the workers and has helped to introduce a culture of patient safety for the majority of professionals involved. Furthermore, the local SNAI facilitates compliance with European regulations regarding the obligation to record incidents in RT.

The Fusion of Incident Learning and Failure Mode and Effects Analysis for Data-Driven Patient Safety Improvements

PURPOSE

Incident learning is a critical part of the quality improvement process for all radiation therapy clinics. Failure mode and effects analysis has also been adopted as a hazard analysis method within the field of radiation oncology based on the recommendations of American Association of Physicists in Medicine Task Group 100. In this work, we demonstrate a fusion of these techniques that is efficient and transferrable to all types of clinics and that allows data-driven targeting of the highest risk error types.

METHODS AND MATERIALS

Four clinical physicists recorded safety events detected during physics treatment plan quality assurance over a 27-month period. Events were sorted into the broad categories of either a documentation or plan construction error. Events were further stratified into subcategories until sufficiently discriminated against for analysis. Event risks were quantified using reduced-resolution TG-100 severity scores combined with observed occurrence rates. The highest risk categories were examined for intervention strategies.

RESULTS

A total of 871 events were identified over the study period. Of these, 652 (74.9%) were classified as low severity, 178 (20.4%) as medium severity, and 41 (4.7%) as high severity. Four of the top 5 ranked categories could be targeted by a preplanning chart rounds. Several of the categories could be targeted by additional automation in the planning and QA processes.

CONCLUSIONS

The retrospective classification and risk analysis of safety events allows clinics to design targeted workflow and quality assurance changes aimed at reducing the occurrence of high-risk events. The method presented here leverages incident learning efforts that many clinics are already performing, allows the severity of events to be efficiently assigned, and generates actionable results without requiring a complete failure mode and effects analysis.

[Analysis of Workflow and Structuring of the Problem Element for Workflow Database Construction of Radiological Technologists in Radiation Therapy]

The work of radiological technologists is changing and more complicated because of the development of medical technology and implementation of information technology (IT). Although the cases of incident and accident have been reported, they have not been comprehensively analyzed in the workflow for radiotherapy. In this study, we visualized the workflow of radiological technologists in radiotherapy and revealed the causes of incidents and accidents. The work process was visualized by drawing workflow map. The structuring of problem was performed with interpretive structural modeling (ISM) method based on graph theory by analyzing of work categorized by safety management. Our results may be able to clarify the work of radiological technologists leads to the reduction of incidents and accidents in radiation therapy.

Incorporating Human Factors Analysis and Classification System (HFACS) Into Analysis of Reported Near Misses and Incidents in Radiation Oncology

PURPOSE

Human factors analysis and classification system (HFACS) is a framework for investigation into causation of human errors. We herein assess whether radiation oncology professionals, with brief training, can conduct HFACS on reported near misses or safety incidents (NMSIs) in a reliable (eg, with a high level of agreement) and practical (eg, timely and with user satisfaction) manner.

METHODS AND MATERIALS

We adapted a classical HFACS framework by selecting and modifying main headings, subheadings, and nano-codes that were most likely to apply to radiation oncology settings. The final modified HFACS included 3 main headings, 8 subheadings, and 20 nano-codes. The modified HFACS was first tested in a simulated trial on 8 NMSI and was analyzed by 5 to 10 radiation oncology professionals, with 2 endpoints: (1) agreement among participants at the main-heading, subheading, and nano-code level, and (2) time to complete the analysis. We then performed a prospective trial integrating this approach into a weekly NMSI review meeting, with 10 NMSIs analyzed by 8 to 13 radiation oncology professionals with the same endpoints, while also collecting survey data on participants' satisfaction.

RESULTS

In the simulated trial, agreement among participants was 85% on the main headings, 73% on the subheadings, and 70% on the nano-codes. Participants needed, on average, 16.4 minutes (standard deviation, 5.7 minutes) to complete an analysis. In the prospective trial, agreement between participants was 81% on the main headings, 75% on the subheadings, and 74% on the nano-codes. Participants needed, on average, 8.3 minutes (standard deviation, 4.7 minutes) to complete an analysis. The average satisfaction with the proposed HFACS approach was 3.9 (standard deviation 1.0) on a scale from 1 to 5.

CONCLUSIONS

This study demonstrates that, after relatively brief training, radiation oncology professionals were able to perform HFACS analysis in a reliable and timely manner and with a relatively high level of satisfaction.

Clinical safety assessment of the Halcyon system

PURPOSE

The Halcyon consists of precommissioned linear accelerator and treatment planning algorithms that were designed to simplify the acceptance, commissioning, and clinical workflow for image-guided intensity-modulated radiotherapy. The purpose of this work was to perform a comprehensive safety assessment for the clinical use of the Halcyon.

METHODS

Systems-Theoretic Process Analysis was used as the safety assessment tool. As part of the analysis, a number of control loops and control actions are created to describe system function. Safety is assessed by determining unsafe control actions and a corresponding list of causal scenarios that leads to accidents. The scope of the analysis was from the acceptance of the Halcyon to routine patient treatments. All aspects of treating patients were considered including the roles of physicians, physicists, dosimetrists, and therapists. The analysis was completed by four physicists with input from other members of the radiation therapy team. The causal scenarios were summarized using the causality categories from the consensus recommendations for incident learning database structures in radiation oncology (Med Phys, Vol. 39, No. 12, Dec 2012).

RESULTS

Twenty-three (23) control loops containing 52 control actions were created for the clinical use of the Halcyon. One hundred forty-four (144) unsafe control actions were identified with 385 associated causal scenarios. Twenty-seven percent (27%) of the causal scenarios were related to equipment technical issues, while 73% of the causal scenarios were predominantly related to procedural issues, human behavior, and organizational management.

CONCLUSIONS

For routine clinical use of closed or largely automated radiation therapy equipment, the majority of safety concerns is related to nontechnical issues. The Halcyon and other similar systems may present opportunities to streamline, reduce, or eliminate some traditional equipment commissioning and routine quality assurance activities in exchange for an increased focus on issues related to organizational management, procedures, and human behavior.

Optimizing efficiency and safety in external beam radiotherapy using automated plan check (APC) tool and six sigma methodology

PURPOSE

To develop and implement an automated plan check (APC) tool using a Six Sigma methodology with the aim of improving safety and efficiency in external beam radiotherapy.

METHODS

The Six Sigma define-measure-analyze-improve-control (DMAIC) framework was used by measuring defects stemming from treatment planning that were reported to the departmental incidence learning system (ILS). The common error pathways observed in the reported data were combined with our departmental physics plan check list, and AAPM TG-275 identified items. Prioritized by risk priority number (RPN) and severity values, the check items were added to the APC tool developed using Varian Eclipse Scripting Application Programming Interface (ESAPI). At 9 months post-APC implementation, the tool encompassed 89 check items, and its effectiveness was evaluated by comparing RPN values and rates of reported errors. To test the efficiency gains, physics plan check time and reported error rate were prospectively compared for 20 treatment plans.

RESULTS

The APC tool was successfully implemented for external beam plan checking. FMEA RPN ranking re-evaluation at 9 months post-APC demonstrated a statistically significant average decrease in RPN values from 129.2 to 83.7 (P < .05). After the introduction of APC, the average frequency of reported treatment-planning errors was reduced from 16.1% to 4.1%. For high-severity errors, the reduction was 82.7% for prescription/plan mismatches and 84.4% for incorrect shift note. The process shifted from 4σ to 5σ quality for isocenter-shift errors. The efficiency study showed a statistically significant decrease in plan check time (10.1 ± 7.3 min, P = .005) and decrease in errors propagating to physics plan check (80%).

CONCLUSIONS

Incorporation of APC tool has significantly reduced the error rate. The DMAIC framework can provide an iterative and robust workflow to improve the efficiency and quality of treatment planning procedure enabling a safer radiotherapy process.

An Audit for Radiotherapy Planning and Treatment Errors From a Low-Middle-Income Country Centre

AIMS

To report the findings of an audit for radiotherapy errors from a low-middle-income country (LMICs) centre. This would serve as baseline data for radiotherapy error rates, their severity and causes, in such centres where modern error reporting and learning processes still do not exist.

MATERIALS AND METHODS

A planned cross-sectional weekly audit of electronic radiotherapy charts at the radiotherapy planning and delivery step for all patients treated with curative intent was conducted. Detailed analysis was carried out to determine the step of origin of error, time and contributing factors. They were graded as per indigenous institutional (TMC) radiotherapy error grading (TREG) system and the contributing factors identified were prioritised using the product of frequency, severity and ease of detection.

RESULTS

In total, 1005 consecutive radically treated patients' charts were audited, 67 radiotherapy errors affecting 60 patients, including 42 incidents and 25 near-misses were identified. Transcriptional errors (29%) were the most common type. Most errors occurred at the time of treatment planning (59.7%), with "plan information transfer to the radiation oncology information system" being the most frequently affected sub-step of the radiotherapy process (47.8%). More errors were noted at cobalt units (52/67; 77.6%) than at linear accelerators. Trend analysis showed an increased number of radiotherapy incidents on Fridays and near-misses on Mondays. Trend for increased radiotherapy errors noted in the evening over other shifts. On severity grading, most of the errors (54/60; 90%) were clinically insignificant (grade I/II). Inadequacies and non-adherence towards standard operating procedures, poor documentation and lack of continuing education were the three most prominent causes.

CONCLUSION

Preliminary data suggest a vulnerability of LMIC set-up to radiotherapy errors and emphasises the need for the development of longitudinal prospective processes, such as voluntary reporting and a continued education system, to ensure robust and comprehensive safe practises on par with centres in developed countries.

Implementation and operation of incident learning across a newly-created health system

PURPOSE

The purpose of this work is to describe our experience launching an expanded incident learning system for patient safety and quality that takes into account aspects beyond therapeutic dose delivery, specifically imaging/simulation incidents, medical care incidents, and operational issues.

METHODS

Our ILS was designed for a newly created health system comprised of a midsized academic hospital and two smaller community hospitals. The main design goal was to create a highly sensitive system to capture as much information throughout the department as possible. Reports were classified according to incidents and near misses involving therapeutic radiation, imaging/simulation, and patient care (not involving radiation), unsafe conditions, operational issues, and accolades/suggestions. Reports were analyzed according to impact on various steps in the process of care. Actions made in response to reports were assessed and characterized by intervention reliability.

RESULTS

A total of 1125 reports were submitted in the first 23 months. For all three departments, therapeutic radiation incidents and near misses consisted of less than one-third of all reports submitted. For the midsized academic department, operational issues and unsafe conditions comprised the largest percentage of reports (70%). Although the majority of reports impacted steps related to the technical aspects of treatment (simulation, planning, and treatment delivery), 20% impacted other steps such as scheduling or clinic visits. More than 160 actions were performed in response to reports. Of these actions, 63 were quality improvement interventions to improve practices, while 97 were learning actions for raising awareness.

CONCLUSIONS

We have developed an ILS that identifies issues related to the entire process of care delivery in radiation oncology, as evidenced by frequent and varied reported events. By identifying a broad spectrum of issues in a department, opportunities for improvement can be identified.

Determining the impact of pre-radiation treatment verification simulation/dry run by analyzing intradepartmental reported incidents and surveying staff and patients

PURPOSE

Error identification in radiation therapy is critical to maintain a safe and efficient therapeutic environment. A verification simulation (VS; also called a dry run for patient information) provides a dedicated time prior to treatment to duplicate steps of patient setup, imaging, and treatment process as a final quality assurance step. Through the use of surveys and analysis of reported incidents, we sought to determine the value of a VS before initiating patient treatment.

METHODS AND MATERIALS

In November 2014, a VS was instituted across our network of 11 radiation oncology clinics. A comparison of the incident rate reported through our departmental incident learning system (ILS) was made between a non-VS group (965 patients who were treated in the 18 months prior to instituting the VS) and a VS group (984 patients who were treated over 18 months with the VS policy in place). From August to December 2016, surveys were completed by 211 patients and 55 physicians, nurses, and therapists detailing their perspectives on the VS.

RESULTS

There were 28 incidents (2.9%) in the non-VS group compared with 18 incidents (1.8%) in the VS group (P = .03). In the VS group, more incidents were detected before the day of treatment (P = .03) and fewer incidents on the day of treatment (P = .02). In addition, a trend toward fewer incidents after treatment started (P = .09) was observed. Patient surveys indicated that 99.5% of patients were informed of the VS, 83% reported decreased anxiety during treatment, and 5% indicated concerns about delaying treatment. The majority of staff members (67%) were satisfied with the VS.

CONCLUSIONS

A VS helps identify and correct incidents before the administration of radiation therapy and reduces patient anxiety.

Risk factors for near-miss events and safety incidents in pediatric radiation therapy

BACKGROUND AND PURPOSE

Factors contributing to safety- or quality-related incidents (e.g. variances) in children are unknown. We identified clinical and RT treatment variables associated with risk for variances in a pediatric cohort.

MATERIALS AND METHODS

Using our institution's incident learning system, 81 patients age ≤21 years old who experienced variances were compared to 191 pediatric patients without variances. Clinical and RT treatment variables were evaluated as potential predictors for variances using univariate and multivariate analyses.

RESULTS

Variances were primarily documentation errors (n = 46, 57%) and were most commonly detected during treatment planning (n = 14, 21%). Treatment planning errors constituted the majority (n = 16 out of 29, 55%) of near-misses and safety incidents (NMSI), which excludes workflow incidents. Therapists reported the majority of variances (n = 50, 62%). Physician cross-coverage (OR = 2.1, 95% CI = 1.04-4.38) and 3D conformal RT (OR = 2.3, 95% CI = 1.11-4.69) increased variance risk. Conversely, age >14 years (OR = 0.5, 95% CI = 0.28-0.88) and diagnosis of abdominal tumor (OR = 0.2, 95% CI = 0.04-0.59) decreased variance risk.

CONCLUSIONS

Variances in children occurred in early treatment phases, but were detected at later workflow stages. Quality measures should be implemented during early treatment phases with a focus on younger children and those cared for by cross-covering physicians.

Patient safety in external beam radiotherapy, results of the ACCIRAD project: Recommendations for radiotherapy institutions and national authorities on assessing risks and analysing adverse error-events and near misses

The ACCIRAD project, commissioned by the European Commission (EC) to develop guidelines for risk analysis of accidental and unintended exposures in external beam radiotherapy (EBRT), was completed in the year 2014. In 2015, the "General guidelines on risk management in external beam radiotherapy" were published as EC report Radiation Protection (RP)-181. The present document is the third and final report of the findings from the ACCIRAD project. The main aim of this paper is to describe the key features of the risk management process and to provide general guidelines for radiotherapy departments and national authorities on risk assessment and analysis of adverse error-events and near misses. The recommendations provided here and in EC report RP-181 are aimed at promoting the harmonisation of risk management systems across Europe, improving patient safety, and enabling more reliable inter-country comparisons.

Application of an incident taxonomy for radiation therapy: Analysis of five years of data from three integrated cancer centres

AIM

To develop and apply a clinical incident taxonomy for radiation therapy.

BACKGROUND

Capturing clinical incident information that focuses on near-miss events is critical for achieving higher levels of safety and reliability.

METHODS AND MATERIALS

A clinical incident taxonomy for radiation therapy was established; coding categories were prescription, consent, simulation, voluming, dosimetry, treatment, bolus, shielding, imaging, quality assurance and coordination of care. The taxonomy was applied to all clinical incidents occurring at three integrated cancer centres for the years 2011-2015. Incidents were managed locally, audited and feedback disseminated to all centres.

RESULTS

Across the five years the total incident rate (per 100 courses) was 8.54; the radiotherapy-specific coded rate was 6.71. The rate of true adverse events (unintended treatment and potential patient harm) was 1.06. Adverse events, where no harm was identified, occurred at a rate of 2.76 per 100 courses. Despite workload increases, overall and actual rates both exhibited downward trends over the 5-year period. The taxonomy captured previously unidentified quality assurance failures; centre-specific issues that contributed to variations in incident trends were also identified.

CONCLUSIONS

The application of a taxonomy developed for radiation therapy enhances incident investigation and facilitates strategic interventions. The practice appears to be effective in our institution and contributes to the safety culture. The ratio of near miss to actual incidents could serve as a possible measure of incident reporting culture and could be incorporated into large scale incident reporting systems.

Artificial Intelligence in Medicine and Radiation Oncology

Artifical Intelligence (AI) was reviewed with a focus on its potential applicability to radiation oncology. The improvement of process efficiencies and the prevention of errors were found to be the most significant contributions of AI to radiation oncology. It was found that the prevention of errors is most effective when data transfer processes were automated and operational decisions were based on logical or learned evaluations by the system. It was concluded that AI could greatly improve the efficiency and accuracy of radiation oncology operations.

Incident learning in radiation oncology: A review

Incident learning is a key component for maintaining safety and quality in healthcare. Its use is well established and supported by professional society recommendations, regulations and accreditation, and objective evidence. There is an active interest in incident learning systems (ILS) in radiation oncology, with over 40 publications since 2010. This article is intended as a comprehensive topic review of ILS in radiation oncology, including history and summary of existing literature, nomenclature and categorization schemas, operational aspects of ILS at the institutional level including event handling and root cause analysis, and national and international ILS for shared learning. Core principles of patient safety in the context of ILS are discussed, including the systems view of error, culture of safety, and contributing factors such as cognitive bias. Finally, the topics of medical error disclosure and second victim syndrome are discussed. In spite of the rapid progress and understanding of ILS, challenges remain in applying ILS to the radiation oncology context. This comprehensive review may serve as a springboard for further work.

Risk factors for radiotherapy incidents: a single institutional experience

We aimed to analyze risk factors for incidents occurring during the practice of external beam radiotherapy (EBRT) at a single Japanese center. Treatment data for EBRT from June 2014 to March 2017 were collected. Data from incident reports submitted during this period were reviewed. Near-miss cases were not included. Risk factors for incidents, including patient characteristics and treatment-related factors, were explored using uni- and multivariate analyses. Factors contributing to each incident were also retrospectively categorized according to the recommendations of the American Association of Physicists in Medicine (AAPM). A total of 2887 patients were treated during the study period, and 26 incidents occurred (0.90% per patient). Previous history of radiotherapy and large fraction size were identified as risk factors for incidents by univariate analysis. Only previous history of radiotherapy was detected as a risk factor in multivariate analysis. Identified categories of contributing factors were human behavior (50.0%), communication (40.6%), and technical (9.4%). The incident rate of EBRT was 0.90% per patient in our institution. Previous history of radiotherapy and large fraction size were detected as risk factors for incidents. Human behavior and communication errors were identified as contributing factors for most incidents.

Development and implementation of a radiation therapy incident learning system compatible with local workflow and a national taxonomy

PURPOSE

Collaborative incident learning initiatives in radiation therapy promise to improve and standardize the quality of care provided by participating institutions. However, the software interfaces provided with such initiatives must accommodate all participants and thus are not optimized for the workflows of individual radiation therapy centers. This article describes the development and implementation of a radiation therapy incident learning system that is optimized for a clinical workflow and uses the taxonomy of the Canadian National System for Incident Reporting - Radiation Treatment (NSIR-RT).

METHODS

The described incident learning system is a novel version of an open-source software called the Safety and Incident Learning System (SaILS). A needs assessment was conducted prior to development to ensure SaILS (a) was intuitive and efficient (b) met changing staff needs and (c) accommodated revisions to NSIR-RT. The core functionality of SaILS includes incident reporting, investigations, tracking, and data visualization. Postlaunch modifications of SaILS were informed by discussion and a survey of radiation therapy staff.

RESULTS

There were 240 incidents detected and reported using SaILS in 2016 and the number of incidents per month tended to increase throughout the year. An increase in incident reporting occurred after switching to fully online incident reporting from an initial hybrid paper-electronic system. Incident templating functionality and a connection with our center's oncology information system were incorporated into the investigation interface to minimize repetitive data entry. A taskable actions feature was also incorporated to document outcomes of incident reports and has since been utilized for 36% of reported incidents.

CONCLUSIONS

Use of SaILS and the NSIR-RT taxonomy has improved the structure of, and staff engagement with, incident learning in our center. Software and workflow modifications informed by staff feedback improved the utility of SaILS and yielded an efficient and transparent solution to categorize incidents with the NSIR-RT taxonomy.

A patient safety education program in a medical physics residency

Education in patient safety and quality of care is a requirement for radiation oncology residency programs according to accrediting agencies. However, recent surveys indicate that most programs lack a formal program to support this learning. The aim of this report was to address this gap and share experiences with a structured educational program on quality and safety designed specifically for medical physics therapy residencies. Five key topic areas were identified, drawn from published recommendations on safety and quality. A didactic component was developed, which includes an extensive reading list supported by a series of lectures. This was coupled with practice-based learning which includes one project, for example, failure modes and effect analysis exercise, and also continued participation in the departmental incident learning system including a root-cause analysis exercise. Performance was evaluated through quizzes, presentations, and reports. Over the period of 2014-2016, five medical physics residents successfully completed the program. Evaluations indicated that the residents had a positive experience. In addition to educating physics residents this program may be adapted for medical physics graduate programs or certificate programs, radiation oncology residencies, or as a self-directed educational project for practicing physicists. Future directions might include a system that coordinates between medical training centers such as a resident exchange program.

A nationwide investigation of radiation therapy event reporting-and-learning systems: Can standards be improved?

INTRODUCTION

Variation exists between event reporting-and-learning systems utilised in radiation therapy. Due to the impact of errors associated with this field of medicine, evidence-based and rigorous systems are imperative. The implementation of such systems facilitates the reactive enhancement of patient safety following an event. The purpose of this study was to evaluate Irish event reporting-and-learning procedures against the current literature using a developed evidence-based process map, and to propose recommendations as to how the national standard could be improved.

METHODS

Radiation Therapy Service Managers of all Irish radiation therapy institutions (n = 12) were invited to participate in an anonymous online questionnaire. Included in the questionnaire was a reporting-and-learning process map developed from evidence-based literature, which was used to assess the institution's practice through the use of vignettes. Frequency analysis of closed-ended questions and thematic analysis of open-ended questions was performed to assess the data.

RESULTS

A 91.7% response rate was achieved. The following areas were found to have the most variation with the evidence-based process map: event classification, external reporting, and dissemination of lessons-learned to a wider audience. Recommendations to standardise practice were made.

CONCLUSION

Opportunities for improvement exist within event reporting-and-learning systems of Irish radiation therapy institutions and recommendations have been made on these. These findings can provide learning for other countries with similar reporting systems.