Human factor associations with safety events in radiation therapy

Clinical effects of re-evaluating a lung SBRT failure mode and effects analysis in a radiotherapy department

PURPOSE

The increasing complexity of radiation treatments can hinder its clinical success. This study aimed to better understand evolving risks by re-evaluating a Failure Mode and Effects Analysis (FMEA) in lung SBRT.

METHODS

An experienced multidisciplinary team conducted an FMEA and made a reassessment 3 years later. A process map was developed with potential failure modes (FMs) identified. High-risk FMs and their possible causes and corrective actions were determined. The initial FMEA analysis was compared to gain a deeper perspective.

RESULTS

We identified 232 FMs. The high-risk processes were plan approval, target contouring, and patient evaluation. The corrective measures were based on stricter standardization of plan approval, pre-planning peer review, and a supporting pretreatment checklist, which substantially reduced the risk priority number in the revised FMEA. In the FMEA reassessment, we observed that the increased complexity and number of patients receiving lung SBRT conditioned a more substantial presence of human factors and communication errors as causal conditions and a potential wrong dose as a final effect.

CONCLUSIONS

Conducting a lung SBRT FMEA analysis has identified high-risk conditions that have been effectively mitigated in an FMEA reanalysis. Plan approval has shown to be a weak link in the process. The increasing complexity of treatments and patient numbers have shifted causal factors toward human failure and communication errors. The potential of a wrong dose as a final effect augments in this scenario. We propose that digital and artificial intelligence options are needed to mitigate potential errors in high-complexity and high-risk RT scenarios.

Causal analysis of radiotherapy safety incidents based on a hybrid model of HFACS and Bayesian network

Objective

This research investigates the role of human factors of all hierarchical levels in radiotherapy safety incidents and examines their interconnections.

Methods

Utilizing the human factor analysis and classification system (HFACS) and Bayesian network (BN) methodologies, we created a BN-HFACS model to comprehensively analyze human factors, integrating the hierarchical structure. We examined 81 radiotherapy incidents from the radiation oncology incident learning system (RO-ILS), conducting a qualitative analysis using HFACS. Subsequently, parametric learning was applied to the derived data, and the prior probabilities of human factors were calculated at each BN-HFACS model level. Finally, a sensitivity analysis was conducted to identify the human factors with the greatest influence on unsafe acts.

Results

The majority of safety incidents reported on RO-ILS were traced back to the treatment planning phase, with skill errors and habitual violations being the primary unsafe acts causing these incidents. The sensitivity analysis highlighted that the condition of the operators, personnel factors, and environmental factors significantly influenced the occurrence of incidents. Additionally, it underscored the importance of organizational climate and organizational process in triggering unsafe acts.

Conclusion

Our findings suggest a strong association between upper-level human factors and unsafe acts among radiotherapy incidents in RO-ILS. To enhance radiation therapy safety and reduce incidents, interventions targeting these key factors are recommended.

Automated detection of vertebral body misalignments in orthogonal kV and MV guided radiotherapy: application to a comprehensive retrospective dataset

Objective. In image-guided radiotherapy (IGRT), off-by-one vertebral body misalignments are rare but potentially catastrophic. In this study, a novel detection method for such misalignments in IGRT was investigated using densely-connected convolutional networks (DenseNets) for applications towards real-time error prevention and retrospective error auditing.Approach. A total of 4213 images acquired from 527 radiotherapy patients aligned with planar kV or MV radiographs were used to develop and test error-detection software modules. Digitally reconstructed radiographs (DRRs) and setup images were retrieved and co-registered according to the clinically applied alignment contained in the DICOM REG files. A semi-automated algorithm was developed to simulate patient positioning errors on the anterior-posterior (AP) and lateral (LAT) images shifted by one vertebral body. A DenseNet architecture was designed to classify either AP images individually or AP and LAT image pairs. Receiver-operator characteristic curves (ROC) and areas under the curves (AUC) were computed to evaluate the classifiers on test subsets. Subsequently, the algorithm was applied to the entire dataset in order to retrospectively determine the absolute off-by-one vertebral body error rate for planar radiograph guided RT at our institution from 2011-2021.Main results. The AUCs for the kV models were 0.98 for unpaired AP and 0.99 for paired AP-LAT. The AUC for the MV AP model was 0.92. For a specificity of 95%, the paired kV model achieved a sensitivity of 99%. Application of the model to the entire dataset yielded a per-fraction off-by-one vertebral body error rate of 0.044% [0.0022%, 0.21%] for paired kV IGRT including one previously unreported error.Significance. Our error detection algorithm was successful in classifying vertebral body positioning errors with sufficient accuracy for retrospective quality control and real-time error prevention. The reported positioning error rate for planar radiograph IGRT is unique in being determined independently of an error reporting system.

Proof-of-concept study of artificial intelligence-assisted review of CBCT image guidance

PURPOSE

Automation and computer assistance can support quality assurance tasks in radiotherapy. Retrospective image review requires significant human resources, and automation of image review remains a noteworthy missing element in previous work. Here, we present initial findings from a proof-of-concept clinical implementation of an AI-assisted review of CBCT registrations used for patient setup.

METHODS

An automated pipeline was developed and executed nightly, utilizing python scripts to interact with the clinical database through DICOM networking protocol and automate data retrieval and analysis. A previously developed artificial intelligence (AI) algorithm scored CBCT setup registrations based on misalignment likelihood, using a scale from 0 (most unlikely) through 1 (most likely). Over a 45-day period, 1357 pre-treatment CBCT registrations from 197 patients were retrieved and analyzed by the pipeline. Daily summary reports of the previous day's registrations were produced. Initial action levels targeted 10% of cases to highlight for in-depth physics review. A validation subset of 100 cases was scored by three independent observers to characterize AI-model performance.

RESULTS

Following an ROC analysis, a global threshold for model predictions of 0.87 was determined, with a sensitivity of 100% and specificity of 82%. Inspecting the observer scores for the stratified validation dataset showed a statistically significant correlation between observer scores and model predictions.

CONCLUSION

In this work, we describe the implementation of an automated AI-analysis pipeline for daily quantitative analysis of CBCT-guided patient setup registrations. The AI-model was validated against independent expert observers, and appropriate action levels were determined to minimize false positives without sacrificing sensitivity. Case studies demonstrate the potential benefits of such a pipeline to bolster quality and safety programs in radiotherapy. To the authors' knowledge, there are no previous works performing AI-assisted assessment of pre-treatment CBCT-based patient alignment.

Comparison between the WHO-CFICPS and the PRISMA classification of safety-related events in a radiation oncology department

INTRODUCTION

Describing Safety-Related Events (SREs) in a radiotherapy (RT) department and comparing WHO-CFICPS (World Health Organization's Conceptual Framework For The International Classification For Patient Safety) and PRISMA (Prevention and Recovery Information System for Monitoring and Analysis) methods for classifying SREs.

METHODS

From February 2017 to October 2020, two Quality Managers (QMs) randomly classified 1173 SREs using 13 incident types of WHO-CFICPS. The same two QMs, reclassified the same SREs according to 20 PRISMA incident codes. Statistical analysis was performed to assess the association between the 13 incident types of WHO-CFICPS and the 20 PRISMA codes. The chi-squared and post-hoc tests using adjusted standardized residuals were applied to detect the association between the two systems.

RESULTS

There was a significant association between WHO-CFICPS incident types and PRISMA codes (P < 0.001). Ninety-two percent of all SREs were categorized using 4 of 13 WHO-CFICPS incident types including Clinical Process/Procedure (n = 448, 38.2%), Clinical Administration (n = 248, 21.1%), Documentation (n = 226, 19.2%) and Resources/Organizational Management (n = 15,613.3%). According to PRISMA classification, 14 of the 20 codes were used to describe the same SREs. PRISMA captured 41 Humans Skill Slips from 226 not better defined WHO-CFICPS Documentation Incidents, 38 Human Rule-based behaviour Qualification from not better defined 447 Clinical Process/Procedure and 40 Organization Management priority events from 156 not better defined WHO-CFICPS Resources/Organizational Management events (P < 0.001).

CONCLUSION

Although there was a significant association between WHO-CFICPS and PRISMA, The PRISMA method provides a more detailed insight into SREs compared to WHO-CFICPS in a RT department.

How Safe Is Cancer Care?

Cancer Morbidity, Mortality, and Improvement Rounds is a series of articles intended to explore the unique safety risks experienced by oncology patients through the lens of quality improvement, systems and human factors engineering, and cognitive psychology. For purposes of clarity, each case focuses on a single theme, although, as is true for all medical incidents, there are almost always multiple, overlapping, contributing factors. The quality improvement paradigm used here, which focuses on root cause analyses and opportunities to improve care delivery systems, was previously outlined in this journal.

Human factor associations with safety events in radiation therapy

BACKGROUND AND PURPOSE

Incident learning can reveal important opportunities for safety improvement, yet learning from error is challenged by a number of human factors. In this study, incident learning reports have been analyzed with the human factors analysis classification system (HFACS) to uncover predictive patterns of human contributing factors.

MATERIALS AND METHODS

Sixteen hundred reports from the Safety in Radiation Oncology incident learning system were filtered for inclusion ultimately yielding 141 reports. A radiotherapy-specific error type was assigned to each event as were all reported human contributing factors. An analysis of associations between human contributing factors and error types was performed.

RESULTS

Multiple associations between human factors were found. Relationships between leadership and risk were demonstrated with supervision failures. Skill-based errors (those done without much thought while performing familiar tasks) were found to pose a significant safety risk to the treatment planning process. Errors made during quality assurance (QA) activities were associated with decision-based errors which indicate lacking knowledge or skills.

CONCLUSION

An application of the HFACS to incident learning reports revealed relationships between human contributing factors and radiotherapy errors. Safety improvement efforts should include supervisory influences as they affect risk and error. An association between skill-based and treatment planning errors showed a need for more mindfulness in this increasingly automated process. An association between decision and QA errors revealed a need for improved education in this area. These and other findings can be used to strategically advance safety.