Predicting Radiation Therapy Process Reliability Using Voluntary Incident Learning System Data

Automated Error Labeling in Radiation Oncology via Statistical Natural Language Processing

A report published in 2000 from the Institute of Medicine revealed that medical errors were a leading cause of patient deaths, and urged the development of error detection and reporting systems. The field of radiation oncology is particularly vulnerable to these errors due to its highly complex process workflow, the large number of interactions among various systems, devices, and medical personnel, as well as the extensive preparation and treatment delivery steps. Natural language processing (NLP)-aided statistical algorithms have the potential to significantly improve the discovery and reporting of these medical errors by relieving human reporters of the burden of event type categorization and creating an automated, streamlined system for error incidents. In this paper, we demonstrate text-classification models developed with clinical data from a full service radiation oncology center (test center) that can predict the broad level and first level category of an error given a free-text description of the error. All but one of the resulting models had an excellent performance as quantified by several metrics. The results also suggest that more development and more extensive training data would further improve future results.

An Investigation of Radiation Treatment Learning Opportunities in Relation to the Radiation Oncology Electronic Medical Record: A Single Institution Experience

Purpose

A modern radiation oncology electronic medical record (RO-EMR) system represents a sophisticated human-computer interface with the potential to reduce human driven errors and improve patient safety. As the RO-EMR becomes an integral part of clinical processes, it may be advantageous to analyze learning opportunities (LO) based on their relationship with the RO-EMR. This work reviews one institution's documented LO to: (1) study their relationship with the RO-EMR workflow, (2) identify best opportunities to improve RO-EMR workflow design, and (3) identify current RO-EMR workflow challenges.

Methods and Materials

Internal LO reports for an 11-year contiguous period were categorized by their relationship to the RO-EMR. We also identify the specific components of the RO-EMR used or involved in each LO. Additionally, contributing factor categories from the ASTRO/AAPM sponsored Radiation Oncology Incident Learning System's (RO-ILS) nomenclature was used to characterize LO directly linked to the RO-EMR.

Results

A total of 163 LO from the 11-year period were reviewed and analyzed. Most (77.2%) LO involved the RO-EMR in some way. The majority of the LO were the results of human/manual operations. The most common RO-EMR components involved in the studied LO were documentation related to patient setup, treatment session schedule functionality, RO-EMR used as a communication/note-delivery tool, and issues with treatment accessories. Most of the LO had staff lack of attention and policy not followed as 2 of the highest occurring contributing factors.

Conclusions

We found that the majority of LO were related to RO-EMR workflow processes. The high-risk areas were related to manual data entry or manual treatment execution. An evaluation of LO as a function of their relationship with the RO-EMR allowed for opportunities for improvement. In addition to regular radiation oncology quality improvement review and policy update, automated functions in RO-EMR remain highly desirable.

Common Error Pathways in CyberKnife™ Radiation Therapy

Purpose/Objectives: Stereotactic radiosurgery (SRS) and stereotactic body radiation therapy (SBRT) may be considered “high risk” due to the high doses per fraction. We analyzed CyberKnife™ (CK) SRS and SBRT-related incidents that were prospectively reported to our in-house incident learning system (ILS) in order to identify severity, contributing factors, and common error pathways.

Material and Methods: From 2012 to 2019, 221 reported incidents related to the 4,569 CK fractions delivered (5.8%) were prospectively analyzed by our multi-professional Quality and Safety Committee with regard to severity, contributing factors, as well as the location where the incident occurred (tripped), where it was discovered (caught), and the safety barriers that were traversed (crossed) on the CK process map. Based on the particular step in the process map that incidents tripped, we categorized incidents into general error pathways.

Results: There were 205 severity grade 1–2 (did not reach patient or no clinical impact), 11 grade 3 (clinical impact unlikely), 5 grade 4 (altered the intended treatment), and 0 grade 5–6 (life-threatening or death) incidents, with human performance being the most common contributing factor (79% of incidents). Incidents most commonly tripped near the time when the practitioner requested CK simulation (e.g., pre-CK simulation fiducial marker placement) and most commonly caught during the physics pre-treatment checklist. The four general error pathways included pre-authorization, billing, and scheduling issues (n= 119); plan quality (n= 30); administration of IV contrast during simulation or pre-medications during treatment (n= 22); and image guidance (n= 12).

Conclusion: Most CK incidents led to little or no patient harm and most were related to billing and scheduling issues. Suboptimal human performance appeared to be the most common contributing factor to CK incidents. Additional study is warranted to develop and share best practices to reduce incidents to further improve patient safety.

Human Error Bowtie Analysis to Enhance Patient Safety in Radiation Oncology

PURPOSE

Ensuring safety within RT is of paramount importance. To further support and augment patient safety efforts, the purpose of this research was to test and refine a robust methodology for analyzing human errors that defeat individual controls within RT quality assurance (QA) programs.

METHODS

The method proposed for performing Bowtie Analysis (BTA) was based on training and recommendations from practitioners in the field of Human Factors and Ergonomics practice. Multidisciplinary meetings to iteratively develop BTA focused on incorrect site setup instructions was conducted.

RESULTS

From November 2015 to February 2017, we had 12 reported incidents related to site setup notes that could have led to site setup errors. Based on this data, we conducted five BTA analyses related to incorrect site setup instructions. None of the individual controls within our QA program designed to check for potential errors with site setup instructions met the level of robustness to be classified as key safeguards or barriers.

CONCLUSIONS

The relatively low number of incidents causing patient harm has led us to typically assume that we have sufficient and effective controls in place to prevent serious human errors from leading to severe patient consequences. Based on our BTA, we question how well we truly understand the details of our individual controls. To meet the level of safety achieved by high reliability organizations (HROs), we need to better ensure that our controls are as reliable and robust as we assume.