The impact of treatment complexity and computer-control delivery technology on treatment delivery errors

Automating QA analysis for a six-degree-of-freedom (6DOF) couch using image displacement and an accelerometer sensor

The purpose of this study is to develop an approach for automating quality assurance (QA) analysis for a six-degree-of-freedom (6DOF) couch using image displacement and an accelerometer sensor. A cubic phantom was fabricated using 3D printing and the accelerometer sensor was embedded in the phantom to measure the couch in the pitch and roll directions. The accuracy and reliability of image displacement and the accelerometer sensor were investigated prior to their practical use for 6DOF couch QA. Image displacement performance had an accuracy and reliability of 0.026 ± 0.026 mm for the translation direction and 0.021 ± 0.016° for the rotation direction. Accelerometer sensor performance had an accuracy and reliability of 0.023 ± 0.018° for pitch rotation and 0.051 ± 0.024° for roll rotation. Automating QA analysis was used to perform 6DOF couch QA, and the couch position errors measured using image displacement were less than 0.99 mm, 0.91 mm, 0.82 mm for the vertical, longitudinal, lateral translation in range between ±20 mm, and 0.07°, 0.23°, and 0.2° for pitch, roll, and yaw rotation in range between ±3° whereas the couch position errors measured using the accelerometer sensor were less than 0.1° and 0.19° for the pitch and roll rotation in range between ±3°.

Impact of technological and departmental changes on incident rates in radiation oncology over a seventeen-year period

INTRODUCTION

Advancements in technology and processes are designed to bring improvement. However, this is often achieved in parallel with increases in complexity, simultaneously presenting opportunities for new types of errors. This study aims to contextualise the impact of internal departmental changes upon radiation incidents and near misses recorded.

METHODS

A timeline of events and a comprehensive incident categorisation system were applied to all radiation incidents and near misses recorded at the Princess Alexandra Hospital Radiation Oncology department from 2003 to 2019, inclusive. Descriptive statistics were performed to identify the type and number of incidents reported during the time period in relation to potential changes within the department, with a focus on the implementation of an electronic environment.

RESULTS

Over the seventeen-year period, 157 incidents and 76 near misses were reported. The majority of incidents were classified as 'procedural' (78%), with 'treatment' being both the highest point of error and point of detection (49% and 85%, respectively). The largest number of incidents and near misses were reported in 2018 (n = 39) which was also a year that experienced the largest number of departmental changes (n = 16), including the move to a completely electronic planning process.

CONCLUSIONS

Changes within the department were followed by an increasing number of reported incidents. Proactive measures should be undertaken prior to the implementation of major changes within the department to aid in the minimisation of incident occurrence.

An Investigation of Radiation Treatment Learning Opportunities in Relation to the Radiation Oncology Electronic Medical Record: A Single Institution Experience

Purpose

A modern radiation oncology electronic medical record (RO-EMR) system represents a sophisticated human-computer interface with the potential to reduce human driven errors and improve patient safety. As the RO-EMR becomes an integral part of clinical processes, it may be advantageous to analyze learning opportunities (LO) based on their relationship with the RO-EMR. This work reviews one institution's documented LO to: (1) study their relationship with the RO-EMR workflow, (2) identify best opportunities to improve RO-EMR workflow design, and (3) identify current RO-EMR workflow challenges.

Methods and Materials

Internal LO reports for an 11-year contiguous period were categorized by their relationship to the RO-EMR. We also identify the specific components of the RO-EMR used or involved in each LO. Additionally, contributing factor categories from the ASTRO/AAPM sponsored Radiation Oncology Incident Learning System's (RO-ILS) nomenclature was used to characterize LO directly linked to the RO-EMR.

Results

A total of 163 LO from the 11-year period were reviewed and analyzed. Most (77.2%) LO involved the RO-EMR in some way. The majority of the LO were the results of human/manual operations. The most common RO-EMR components involved in the studied LO were documentation related to patient setup, treatment session schedule functionality, RO-EMR used as a communication/note-delivery tool, and issues with treatment accessories. Most of the LO had staff lack of attention and policy not followed as 2 of the highest occurring contributing factors.

Conclusions

We found that the majority of LO were related to RO-EMR workflow processes. The high-risk areas were related to manual data entry or manual treatment execution. An evaluation of LO as a function of their relationship with the RO-EMR allowed for opportunities for improvement. In addition to regular radiation oncology quality improvement review and policy update, automated functions in RO-EMR remain highly desirable.

Electronic charting of radiation therapy planning and treatment: Report of Task Group 262

While most Radiation Oncology clinics have adopted electronic charting in one form or another, no consensus document exists that provides guidelines for safe and effective use of the Radiation Oncology electronic medical records (RO-EMR). Task Group 262 was formed to provide these guidelines as well as to provide recommendations to vendors for improving electronic charting functionality in future. Guidelines are provided in the following areas: Implementation and training for the RO-EMR, acceptance testing and quality assurance (QA) of the RO-EMR, use of the RO-EMR as an information repository, use of the RO-EMR as a workflow manager, electronic charting for brachytherapy and nonstandard treatments, and information technology (IT) considerations associated with the RO-EMR. The report was based on a literature search by the task group, an extensive survey of task group members on their respective RO-EMR practices, an AAPM membership survey on electronic charting, as well as group consensus.

Common Error Pathways in CyberKnife™ Radiation Therapy

Purpose/Objectives: Stereotactic radiosurgery (SRS) and stereotactic body radiation therapy (SBRT) may be considered “high risk” due to the high doses per fraction. We analyzed CyberKnife™ (CK) SRS and SBRT-related incidents that were prospectively reported to our in-house incident learning system (ILS) in order to identify severity, contributing factors, and common error pathways.

Material and Methods: From 2012 to 2019, 221 reported incidents related to the 4,569 CK fractions delivered (5.8%) were prospectively analyzed by our multi-professional Quality and Safety Committee with regard to severity, contributing factors, as well as the location where the incident occurred (tripped), where it was discovered (caught), and the safety barriers that were traversed (crossed) on the CK process map. Based on the particular step in the process map that incidents tripped, we categorized incidents into general error pathways.

Results: There were 205 severity grade 1–2 (did not reach patient or no clinical impact), 11 grade 3 (clinical impact unlikely), 5 grade 4 (altered the intended treatment), and 0 grade 5–6 (life-threatening or death) incidents, with human performance being the most common contributing factor (79% of incidents). Incidents most commonly tripped near the time when the practitioner requested CK simulation (e.g., pre-CK simulation fiducial marker placement) and most commonly caught during the physics pre-treatment checklist. The four general error pathways included pre-authorization, billing, and scheduling issues (n= 119); plan quality (n= 30); administration of IV contrast during simulation or pre-medications during treatment (n= 22); and image guidance (n= 12).

Conclusion: Most CK incidents led to little or no patient harm and most were related to billing and scheduling issues. Suboptimal human performance appeared to be the most common contributing factor to CK incidents. Additional study is warranted to develop and share best practices to reduce incidents to further improve patient safety.

Using failure mode and effects analysis (FMEA) to generate an initial plan check checklist for improved safety in radiation treatment

PURPOSE

To apply failure mode and effect analysis (FMEA) to generate an effective and efficient initial physics plan checklist.

METHODS

A team of physicists, dosimetrists, and therapists was setup to reconstruct the workflow processes involved in the generation of a treatment plan beginning from simulation. The team then identified possible failure modes in each of the processes. For each failure mode, the severity (S), frequency of occurrence (O), and the probability of detection (D) was assigned a value and the risk priority number (RPN) was calculated. The values assigned were based on TG 100. Prior to assigning a value, the team discussed the values in the scoring system to minimize randomness in scoring. A local database of errors was used to help guide the scoring of frequency.

RESULTS

Twenty-seven process steps and 50 possible failure modes were identified starting from simulation to the final approved plan ready for treatment at the machine. Any failure mode that scored an average RPN value of 20 or greater was deemed "eligible" to be placed on the second checklist. In addition, any failure mode with a severity score value of 4 or greater was also considered for inclusion in the checklist. As a by-product of this procedure, safety improvement methods such as automation and standardization of certain processes (e.g., dose constraint checking, check tools), removal of manual transcription of treatment-related information as well as staff education were implemented, although this was not the team's original objective. Prior to the implementation of the new FMEA-based checklist, an in-service for all the second checkers was organized to ensure further standardization of the process.

CONCLUSION

The FMEA proved to be a valuable tool for identifying vulnerabilities in our workflow and processes in generating a treatment plan and subsequently a new, more effective initial plan checklist was created.

Development and validation of a medication regimen complexity scoring tool for critically ill patients

PURPOSE

The purpose of this study was to develop and validate a novel medication regimen complexity-intensive care unit (MRC-ICU) scoring tool in critically ill patients and to correlate MRC with illness severity and patient outcomes.

METHODS

This study was a single-center, retrospective observational chart review of adults admitted to the medical ICU (MICU) between November 2016 and June 2017. The primary aim was the development and internal validation of the MRC-ICU scoring tool. Secondary aims included external validation of the MRC-ICU and exploration of relationships between medication regimen complexity and patient outcomes. Exclusion criteria included a length of stay of less than 24 hours in the MICU, active transfer, or hospice orders at 24 hours. A total of 130 patient medication regimens were used to test, modify, and validate the MRC-ICU tool.

RESULTS

The 39-line item medication regimen complexity scoring tool was validated both internally and externally. Convergent validity was confirmed with total medications (p < 0.0001). Score discriminant validity was confirmed by lack of association with age (p = 0.1039) or sex (p = 0.7829). The MRC-ICU score was significantly associated with ICU length of stay (p = 0.0166), ICU mortality (p = 0.0193), and patient acuity (p < 0.0001).

CONCLUSION

The MRC-ICU scoring tool was validated and found to correlate with length of stay, inpatient mortality, and patient acuity.

A Usability Study of 3 Radiotherapy Systems: A Comparative Evaluation Based on Expert Evaluation and User Experience

BACKGROUND The complex user interface design of radiotherapy treatment delivery systems can lead to use error and patient harm. In this study, we present the results of a comparison of 3 radiotherapy treatment delivery systems now used in China. MATERIAL AND METHODS We conducted a comprehensive usability study of 3 radiotherapy treatment delivery systems. Expert evaluation was performed through heuristic evaluation with 3 human-factors experts and 1 experienced radiation therapist for each system. User experience was assessed through perceived system usability and workload, using the National Aeronautics and Space Administration Task Load Index and the Post-Study System Usability Questionnaire. RESULTS For the expert evaluation, 47 usability problems were identified for Varian Trilogy, 75 for Elekta Precise, and 37 for Shinva XHA600E. Most problems were classified as major and minor usability problems, and were found in the process of patient setup and setup verification. For the user experience, radiation therapists presented a lower workload for Varian Trilogy compared to Elekta Precise (P<0.01) and Shinva XHA600E (P<0.01), and a lower workload for Elekta Precise compared to Shinva XHA600E (P=0.020). Radiation therapists perceived a higher system usability for Varian Trilogy compared to Shinva XHA600E (P<0.01), and a higher system usability for Elekta Precise compared to Shinva XHA600E (P<0.01). CONCLUSIONS This research provides valuable data on how 3 radiotherapy treatment delivery systems compare. The results of this study may be useful for hospital equipment procurement decisions, and designing next-generation products to improve patient safety.

Application of an incident taxonomy for radiation therapy: Analysis of five years of data from three integrated cancer centres

AIM

To develop and apply a clinical incident taxonomy for radiation therapy.

BACKGROUND

Capturing clinical incident information that focuses on near-miss events is critical for achieving higher levels of safety and reliability.

METHODS AND MATERIALS

A clinical incident taxonomy for radiation therapy was established; coding categories were prescription, consent, simulation, voluming, dosimetry, treatment, bolus, shielding, imaging, quality assurance and coordination of care. The taxonomy was applied to all clinical incidents occurring at three integrated cancer centres for the years 2011-2015. Incidents were managed locally, audited and feedback disseminated to all centres.

RESULTS

Across the five years the total incident rate (per 100 courses) was 8.54; the radiotherapy-specific coded rate was 6.71. The rate of true adverse events (unintended treatment and potential patient harm) was 1.06. Adverse events, where no harm was identified, occurred at a rate of 2.76 per 100 courses. Despite workload increases, overall and actual rates both exhibited downward trends over the 5-year period. The taxonomy captured previously unidentified quality assurance failures; centre-specific issues that contributed to variations in incident trends were also identified.

CONCLUSIONS

The application of a taxonomy developed for radiation therapy enhances incident investigation and facilitates strategic interventions. The practice appears to be effective in our institution and contributes to the safety culture. The ratio of near miss to actual incidents could serve as a possible measure of incident reporting culture and could be incorporated into large scale incident reporting systems.

Implementation of an Integrated “Record and Verify” System for Data and Images in Radiotherapy

In November 1999, new radiotherapy and medical physics departments began to operate in our institution. In the departments' project, considerable emphasis had been placed on the integration of the equipment through networking. In particular, a “record and verify” (R&V) system for data and images (Varian “Varis” and “Vision”) has been employed in the daily routine since the beginning of treatments. After one year, more than 600 patients have been treated, and a first evaluation of the system can be carried out. In clinical practice, the software required no significant changes in the already-established radiotherapy workflow and has been successfully configured in an acceptable time. On the other hand, the way information is exchanged and shared has dramatically changed towards a nearly paperless and filmless way of working. A good application reliability has been experienced, and the client-server structure has been proven useful by avoiding loss of data in critical situations. The staff time spent for the application to run is now well balanced by the increased safety in daily practice, and digital archiving of all the radiotherapy activity has increased the effectiveness of routine tasks and statistical researches. The critical aspects we have experienced in the application are the proper user rights definition and a minor rounding error in the daily dose accumulation. Future developments in implementation of the system will be aimed to a better integration with the Hospital Information System and to a further exploitation of the RT-PACS features of the “Vision” application.

Incident learning in radiation oncology: A review

Incident learning is a key component for maintaining safety and quality in healthcare. Its use is well established and supported by professional society recommendations, regulations and accreditation, and objective evidence. There is an active interest in incident learning systems (ILS) in radiation oncology, with over 40 publications since 2010. This article is intended as a comprehensive topic review of ILS in radiation oncology, including history and summary of existing literature, nomenclature and categorization schemas, operational aspects of ILS at the institutional level including event handling and root cause analysis, and national and international ILS for shared learning. Core principles of patient safety in the context of ILS are discussed, including the systems view of error, culture of safety, and contributing factors such as cognitive bias. Finally, the topics of medical error disclosure and second victim syndrome are discussed. In spite of the rapid progress and understanding of ILS, challenges remain in applying ILS to the radiation oncology context. This comprehensive review may serve as a springboard for further work.

Quality Assurance of a Record-and-Verify System

Aims and background

With the introduction of more complex three-dimensional conformal radiotherapy and intensity-modulated radiotherapy techniques in clinical practice, the use of record-and-verify systems is recommended to improve the accuracy of radiotherapy treatments. The aim of the present study was to evaluate, for a commercial record-and-verify system, the efficiency, integration with the treatment planning system, and impact of manual checking of data. The most frequent errors or misses were also evaluated.

Materials and methods

The development of internal protocols to systematically implement new technologies has been identified as a priority in the departmental quality assurance process. Data electronically fed into the record-and-verify system were compared with those manually recorded in the clinical paper chart over a period of almost 6 years (October 2000 to December 2006). A total of 7768 treated patients was reviewed. The check was performed by using a homemade data base in which the errors are stratified as follows: 1) general section, 2) geometric and dosimetric section, and 3) delivered dose section.

Results

On a total of 7768 checked patients, one or more mismatches between treatment planning system data and record-and-verify system data or paper chart data were observed for 452 patients (5.8% of total number of inspected patients). The percentage of discrepancies out of the total was: 2.2% in the general section, 3.3% in the dosimetric and geometric section, and 4.2% in the delivered-dose section.

Conclusions

Although record-and-verify systems assume a crucial role in the accuracy and reproducibility of radiation treatment, their inability to eradicate all the errors requires vigilance on the part of the radiation therapy and physics team.

Nature of Medical Malpractice Claims Against Radiation Oncologists

PURPOSE

To examine characteristics of medical malpractice claims involving radiation oncologists closed during a 10-year period.

METHODS AND MATERIALS

Malpractice claims filed against radiation oncologists from 2003 to 2012 collected by a nationwide liability insurance trade association were analyzed. Outcomes included the nature of claims and indemnity payments, including associated presenting diagnoses, procedures, alleged medical errors, and injury severity. We compared the likelihood of a claim resulting in payment in relation to injury severity categories (death as referent) using binomial logistic regression.

RESULTS

There were 362 closed claims involving radiation oncology, 102 (28%) of which were paid, resulting in $38 million in indemnity payments. The most common alleged errors included "improper performance" (38% of closed claims, 18% were paid; 29% [$11 million] of total indemnity), "errors in diagnosis" (25% of closed claims, 46% were paid; 44% [$17 million] of total indemnity), and "no medical misadventure" (14% of closed claims, 8% were paid; less than 1% [$148,000] of total indemnity). Another physician was named in 32% of claims, and consent issues/breach of contract were cited in 18%. Claims for injury resulting in death represented 39% of closed claims and 25% of total indemnity. "Improper performance" was the primary alleged error associated with injury resulting in death. Compared with claims involving death, major temporary injury (odds ratio [OR] 2.8, 95% confidence interval [CI] 1.29-5.85, P=.009), significant permanent injury (OR 3.1, 95% CI 1.48-6.46, P=.003), and major permanent injury (OR 5.5, 95% CI 1.89-16.15, P=.002) had a higher likelihood of a claim resulting in indemnity payment.

CONCLUSIONS

Improper performance was the most common alleged malpractice error. Claims involving significant or major injury were more likely to be paid than those involving death. Insights into the nature of liability claims against radiation oncologists may help direct efforts to improve quality of care and minimize the risk of being sued.

The report of Task Group 100 of the AAPM: Application of risk analysis methods to radiation therapy quality management

The increasing complexity of modern radiation therapy planning and delivery challenges traditional prescriptive quality management (QM) methods, such as many of those included in guidelines published by organizations such as the AAPM, ASTRO, ACR, ESTRO, and IAEA. These prescriptive guidelines have traditionally focused on monitoring all aspects of the functional performance of radiotherapy (RT) equipment by comparing parameters against tolerances set at strict but achievable values. Many errors that occur in radiation oncology are not due to failures in devices and software; rather they are failures in workflow and process. A systematic understanding of the likelihood and clinical impact of possible failures throughout a course of radiotherapy is needed to direct limit QM resources efficiently to produce maximum safety and quality of patient care. Task Group 100 of the AAPM has taken a broad view of these issues and has developed a framework for designing QM activities, based on estimates of the probability of identified failures and their clinical outcome through the RT planning and delivery process. The Task Group has chosen a specific radiotherapy process required for "intensity modulated radiation therapy (IMRT)" as a case study. The goal of this work is to apply modern risk-based analysis techniques to this complex RT process in order to demonstrate to the RT community that such techniques may help identify more effective and efficient ways to enhance the safety and quality of our treatment processes. The task group generated by consensus an example quality management program strategy for the IMRT process performed at the institution of one of the authors. This report describes the methodology and nomenclature developed, presents the process maps, FMEAs, fault trees, and QM programs developed, and makes suggestions on how this information could be used in the clinic. The development and implementation of risk-assessment techniques will make radiation therapy safer and more efficient.

SafetyNet: Streamlining and Automating QA in radiotherapy

Proper quality assurance (QA) of the radiotherapy process can be time‐consuming and expensive. Many QA efforts, such as data export and import, are inefficient when done by humans. Additionally, humans can be unreliable, lose attention, and fail to complete critical steps that are required for smooth operations. In our group we have sought to break down the QA tasks into separate steps and to automate those steps that are better done by software running autonomously or at the instigation of a human. A team of medical physicists and software engineers worked together to identify opportunities to streamline and automate QA. Development efforts follow a formal cycle of writing software requirements, developing software, testing and commissioning. The clinical release process is separated into clinical evaluation testing, training, and finally clinical release. We have improved six processes related to QA and safety. Steps that were previously performed by humans have been automated or streamlined to increase first‐time quality, reduce time spent by humans doing low‐level tasks, and expedite QA tests. Much of the gains were had by automating data transfer, implementing computer‐based checking and automation of systems with an event‐driven framework. These coordinated efforts by software engineers and clinical physicists have resulted in speed improvements in expediting patient‐sensitive QA tests.

PACS number(s): 87.55.Ne, 87.55.Qr, 87.55.tg, 87.55.tm

Evaluating the Impact of In Vivo EPID Dosimetry on Intensity-Modulated Radiation Therapy Treatment Delivery Workflow: A Stakeholder Perspective

INTRODUCTION

In vivo electronic portal imaging device (EPID) dosimetry is an advanced imaging technique that can obtain patient-specific dose data for quality assurance purposes. However, clinical integration of this technique remains a challenge. This study evaluates the impact of implementing an in vivo EPID technique into the treatment delivery workflow for head and neck cancer (HNC) patients in a large cancer centre setting.

MATERIALS/METHODS

Intensity-modulated radiation therapy treatment delivery was simulated on a phantom for 10 HNC cases with and without in vivo EPID dosimetry. Investigators performed the EPID technique by using a preliminary protocol written by medical physicists. Process maps were created to illustrate changes in treatment delivery workflow.

RESULTS

Treatment delivery times increased by an average of 2.34 minutes (P = .0006) when the EPID technique was used. Factors that increased treatment times included the time for storing captured EPID data, adjustment of the imaging panel position as a function of field size, and an inability to use automatic field sequencing when acquiring images.

CONCLUSIONS

The involvement of stakeholders in protocol development allows for the identification of usability issues and staff training needs. Findings from this study have identified limitations of the in vivo EPID technique that may negatively impact treatment delivery workflow. Efficiencies within in vivo EPID dosimetry systems can be improved by enabling automatic field sequencing with automatic image-saving capabilities.

Application of failure mode and effects analysis to intracranial stereotactic radiation surgery by linear accelerator

PURPOSE

The aim of this study was to analyze the application of the failure modes and effects analysis (FMEA) to intracranial stereotactic radiation surgery (SRS) by linear accelerator in order to identify the potential failure modes in the process tree and adopt appropriate safety measures to prevent adverse events (AEs) and near-misses, thus improving the process quality.

METHODS AND MATERIALS

A working group was set up to perform FMEA for intracranial SRS in the framework of a quality assurance program. FMEA was performed in 4 consecutive tasks: (1) creation of a visual map of the process; (2) identification of possible failure modes; (3) assignment of a risk probability number (RPN) to each failure mode based on tabulated scores of severity, frequency of occurrence and detectability; and (4) identification of preventive measures to minimize the risk of occurrence.

RESULTS

The whole SRS procedure was subdivided into 73 single steps; 116 total possible failure modes were identified and a score of severity, occurrence, and detectability was assigned to each. Based on these scores, RPN was calculated for each failure mode thus obtaining values from 1 to 180. In our analysis, 112/116 (96.6%) RPN values were <60, 2 (1.7%) between 60 and 125 (63, 70), and 2 (1.7%) >125 (135, 180). The 2 highest RPN scores were assigned to the risk of using the wrong collimator's size and incorrect coordinates on the laser target localizer frame.

CONCLUSION

Failure modes and effects analysis is a simple and practical proactive tool for systematic analysis of risks in radiation therapy. In our experience of SRS, FMEA led to the adoption of major changes in various steps of the SRS procedure.

A comprehensive quality assurance program for personnel and procedures in radiation oncology: value of voluntary error reporting and checklists

PURPOSE

This report describes the value of a voluntary error reporting system and the impact of a series of quality assurance (QA) measures including checklists and timeouts on reported error rates in patients receiving radiation therapy.

METHODS AND MATERIALS

A voluntary error reporting system was instituted with the goal of recording errors, analyzing their clinical impact, and guiding the implementation of targeted QA measures. In response to errors committed in relation to treatment of the wrong patient, wrong treatment site, and wrong dose, a novel initiative involving the use of checklists and timeouts for all staff was implemented. The impact of these and other QA initiatives was analyzed.

RESULTS

From 2001 to 2011, a total of 256 errors in 139 patients after 284,810 external radiation treatments (0.09% per treatment) were recorded in our voluntary error database. The incidence of errors related to patient/tumor site, treatment planning/data transfer, and patient setup/treatment delivery was 9%, 40.2%, and 50.8%, respectively. The compliance rate for the checklists and timeouts initiative was 97% (P<.001). These and other QA measures resulted in a significant reduction in many categories of errors. The introduction of checklists and timeouts has been successful in eliminating errors related to wrong patient, wrong site, and wrong dose.

CONCLUSIONS

A comprehensive QA program that regularly monitors staff compliance together with a robust voluntary error reporting system can reduce or eliminate errors that could result in serious patient injury. We recommend the adoption of these relatively simple QA initiatives including the use of checklists and timeouts for all staff to improve the safety of patients undergoing radiation therapy in the modern era.

The effect of failure mode and effect analysis on reducing percutaneous coronary intervention hospital door-to-balloon time and mortality in ST segment elevation myocardial infarction

BACKGROUND

Door-to-balloon (D2B) time is an important factor in the outcome of ST segment elevation myocardial infarction (STEMI) treated with primary percutaneous coronary intervention. We aimed to use failure mode and effect analysis to reduce the D2B time for patients with STEMI and to improve clinical outcomes.

METHODS

There were three stages in this study. In Stage 0, data collected from 2005-2006 was used to identify failures in the process, and during Stage 2 (2007) and Stage 3 (2008) the efficacy of intrahospital and interhospital strategies to reduce the D2B time were evaluated. This study enrolled 385 patients; 86 from 2005-2006; 80 in 2007; and 219 in 2008.

RESULTS

By making improvements in these steps, the median D2B time was reduced from 146 min to 32 min for all patients. The proportion of patients with a D2B time of <90 min significantly increased from Stage 0 to Stage 1 and from Stage 1 to Stage 2, for all patients as well as for the non-transferred and transferred subgroups of patients (all p values <0.0001). For non-transferred patients, only reinfarction-free survival showed significant difference among the three stages (p=0.0225), and for transferred patients, only overall survival showed significant difference among the three stages (p=0.0322). Cox's proportional hazards regression analysis showed Stage 2 was associated with a lower risk of reinfarction and mortality compared with Stage 0.

CONCLUSIONS

This study found that failure mode and effect analysis is a powerful method for identifying weaknesses in D2B processes and evaluating strategies to reduce the D2B time.

Pretreatment verification of dose calculation and delivery by means of measurements with PLEXITOM™ phantom

AIM

To validate a pretreatment verification method of dose calculation and dose delivery based on measurements with Metaplex PTW phantom.

BACKGROUND

The dose-response relationships for local tumor control and radiosensitive tissue complications are strong. It is widely accepted that an accuracy of dose delivery of about 3.5% (one standard deviation) is required in modern radiotherapy. This goal is difficult to achieve. This paper describes our experience with the control of dose delivery and calculations at the ICRU reference point.

MATERIALS AND METHODS

The calculations of dose at the ICRU reference point performed with the treatment planning system CMS XiO were checked by measurements carried out in the PLEXITOM™ phantom. All measurements were performed with the ion chamber positioned in the phantom, at the central axis of the beam, at depth equivalent to the radiological depth (at gantry zero position). The source-to-phantom surface distance was always set to keep the source-to-detector distance equal to the reference point depth defined in the ICRU Report 50 (generally, 100 cm). The dose was measured according to IAEA TRS 398 report for measurements in solid phantoms. The measurement results were corrected with the actual accelerator's output factor and for the non-full scatter conditions. Measurements were made for 111 patients and 327 fields.

RESULTS

The average differences between measurements and calculations were 0.03% (SD = 1.4%), 0.3% (SD = 1.0%), 0.1% (SD = 1.1%), 0.6% (SD = 1.8%), 0.3% (SD = 1.5%) for all measurements, for total dose, for pelvis, thorax and H&N patients, respectively. Only in 15 cases (4.6%), the difference between the measured and the calculated dose was greater than 3%. For these fields, a detailed analysis was undertaken.

CONCLUSION

The verification method provides an instantaneous verification of dose calculations before the beginning of a patient's treatment. It allows to detect differences smaller than 3.5%.

Technical note: electronic chart checks in a paperless radiation therapy clinic

PURPOSE

EcCk, which stands for Electronic Chart ChecK, is a computer software and database system. It was developed to improve quality and efficiency of patient chart checking in radiation oncology departments. The core concept is to automatically collect and analyze patient treatment data, and to report discrepancies and potential concerns.

METHODS

EcCk consists of several different computer technologies, including relational database, DICOM, dynamic HTML, and image processing. Implemented in MATLAB and C#, EcCk processes patient data in DICOM, PDF, Microsoft Word, database, and Pinnacle native formats. Generated reports are stored on the storage server and indexed in the database. A standalone report-browser program is implemented to allow users to view reports on any computer in the department. Checks are performed according to predefined logical rules, and results are presented through color-coded reports in which discrepancies are summarized and highlighted. Users examine the reports and take appropriate actions. The core design is intended to automate human task and to improve the reliability of the performed tasks. The software is not intended to replace human audits but rather to aid as a decision support tool.

RESULTS

The software was successfully implemented in the clinical environment and has demonstrated the feasibility of automation of this common task with modern clinical tools. The software integrates multiple disconnected systems and successfully supports analysis of data in diverse formats.

CONCLUSIONS

While the human is the ultimate expert, EcCk has a significant potential to improve quality and efficiency of patient treatment record audits, and to allow verification of tasks that are not easily performed by humans. EcCk can potentially relieve human experts from simple and repetitive tasks, and allow them to work on other important tasks, and in the end to improve the quality and safety of radiation therapy treatments.

Impact of complexity and computer control on errors in radiation therapy

A number of recent publications in both the lay and scientific press have described major errors in patient radiation treatments, and this publicity has galvanised much work to address and mitigate potential safety issues throughout the radiation therapy planning and delivery process. The complexity of modern radiotherapy techniques and equipment, including computer-controlled treatment machines and treatment management systems, as well as sophisticated treatment techniques that involve intensity-modulated radiation therapy, image-guided radiation therapy, stereotactic body radiation therapy, volumetric modulated arc therapy, respiratory gating, and others, leads to concern about safety issues related to that complexity. This article illustrates the relationship between complexity and computer control, and various safety problems and errors that have been reported, and describes studies that address the issue of these modern techniques and whether their complexity does, in fact, result in more errors or safety-related problems. Clinical implications of these results are discussed, as are some of the ways in which the field should respond to the ongoing concerns about errors and complexity in radiation therapy.

Daily orthogonal kilovoltage imaging using a gantry-mounted on-board imaging system results in a reduction in radiation therapy delivery errors

PURPOSE

To determine whether the use of routine image guided radiation therapy (IGRT) using pretreatment on-board imaging (OBI) with orthogonal kilovoltage X-rays reduces treatment delivery errors.

METHODS AND MATERIALS

A retrospective review of documented treatment delivery errors from 2003 to 2009 was performed. Following implementation of IGRT in 2007, patients received daily OBI with orthogonal kV X-rays prior to treatment. The frequency of errors in the pre- and post-IGRT time frames was compared. Treatment errors (TEs) were classified as IGRT-preventable or non-IGRT-preventable.

RESULTS

A total of 71,260 treatment fractions were delivered to 2764 patients. A total of 135 (0.19%) TEs occurred in 39 (1.4%) patients (3.2% in 2003, 1.1% in 2004, 2.5% in 2005, 2% in 2006, 0.86% in 2007, 0.24% in 2008, and 0.22% in 2009). In 2007, the TE rate decreased by >50% and has remained low (P = .00007, compared to before 2007). Errors were classified as being potentially preventable with IGRT (e.g., incorrect site, patient, or isocenter) vs. not. No patients had any IGRT-preventable TEs from 2007 to 2009, whereas there were 9 from 2003 to 2006 (1 in 2003, 2 in 2004, 2 in 2005, and 4 in 2006; P = .0058) before the implementation of IGRT.

CONCLUSIONS

IGRT implementation has a patient safety benefit with a significant reduction in treatment delivery errors. As such, we recommend the use of IGRT in routine practice to complement existing quality assurance measures.

Six sigma tools for a patient safety-oriented, quality-checklist driven radiation medicine department

INTRODUCTION

The purpose of this work was to develop and implement six sigma practices toward the enhancement of patient safety in an electronic, quality checklist-driven, multicenter, paperless radiation medicine department.

METHODS AND MATERIALS

A quality checklist process map (QPM), stratified into consultation through treatment-completion stages was incorporated into an oncology information systems platform. A cross-functional quality management team conducted quality-function-deployment and define-measure-analyze-improve-control (DMAIC) six sigma exercises with a focus on patient safety. QPM procedures were Pareto-sorted in order of decreasing patient safety risk with failure mode and effects analysis (FMEA). Quantitative metrics for a grouped set of highest risk procedures were established. These included procedural delays, associated standard deviations and six sigma Z scores. Baseline performance of the QPM was established over the previous year of usage. Data-driven analysis led to simplification, standardization, and refinement of the QPM with standard deviation, slip-day reduction, and Z-score enhancement goals. A no-fly policy (NFP) for patient safety was introduced at the improve-control DMAIC phase, with a process map interlock imposed on treatment initiation in the event of FMEA-identified high-risk tasks being delayed or not completed. The NFP was introduced in a pilot phase with specific stopping rules and the same metrics used for performance assessments. A custom root-cause analysis database was deployed to monitor patient safety events.

RESULTS

Relative to the baseline period, average slip days and standard deviations for the risk-enhanced QPM procedures improved by over threefold factors in the NFP period. The Z scores improved by approximately 20%. A trend for proactive delays instead of reactive hard stops was observed with no adverse effects of the NFP. The number of computed potential no-fly delays per month dropped from 60 to 20 over a total of 520 cases. The fraction of computed potential no-fly cases that were delayed in NFP compliance rose from 28% to 45%. Proactive delays rose to 80% of all delayed cases. For potential no-fly cases, event reporting rose from 18% to 50%, while for actually delayed cases, event reporting rose from 65% to 100%.

CONCLUSIONS

With complex technologies, resource-compromised staff, and pressures to hasten treatment initiation, the use of the six sigma driven process interlocks may mitigate potential patient safety risks as demonstrated in this study.

Applying usability heuristics to radiotherapy systems

BACKGROUND AND PURPOSE

Heuristic evaluations have been used to evaluate safety of medical devices by identifying and assessing usability issues. Since radiotherapy treatment delivery systems often consist of multiple complex user-interfaces, a heuristic evaluation was conducted to assess the potential safety issues of such a system.

MATERIAL AND METHODS

A heuristic evaluation was conducted to evaluate the treatment delivery system at Princess Margaret Hospital (Toronto, Canada). Two independent evaluators identified usability issues with the user-interfaces and rated the severity of each issue.

RESULTS

The evaluators identified 75 usability issues in total. Eighteen of them were rated as high severity, indicating the potential to have a major impact on patient safety. A majority of issues were found on the record and verify system, and many were associated with the patient setup process. While the hospital has processes in place to ensure patient safety, recommendations were developed to further mitigate the risks of potential consequences.

CONCLUSIONS

Heuristic evaluation is an efficient and inexpensive method that can be successfully applied to radiotherapy delivery systems to identify usability issues and improve patient safety. Although this study was conducted only at one site, the findings may have broad implications for the design of these systems.

Implementation of electronic checklists in an oncology medical record: initial clinical experience

PURPOSE

The quality of any medical treatment depends on the accurate processing of multiple complex components of information, with proper delivery to the patient. This is true for radiation oncology, in which treatment delivery is as complex as a surgical procedure but more dependent on hardware and software technology. Uncorrected errors, even if small or infrequent, can result in catastrophic consequences for the patient. We developed electronic checklists (ECLs) within the oncology electronic medical record (EMR) and evaluated their use and report on our initial clinical experience.

METHODS

Using the Mosaiq EMR, we developed checklists within the clinical assessment section. These checklists are based on the process flow of information from one group to another within the clinic and enable the processing, confirmation, and documentation of relevant patient information before the delivery of radiation therapy. The clinical use of the ECL was documented by means of a customized report.

RESULTS

Use of ECL has reduced the number of times that physicians were called to the treatment unit. In particular, the ECL has ensured that therapists have a better understanding of the treatment plan before the initiation of treatment. An evaluation of ECL compliance showed that, with additional staff training, > 94% of the records were completed.

CONCLUSION

The ECL can be used to ensure standardization of procedures and documentation that the pretreatment checks have been performed before patient treatment. We believe that the implementation of ECLs will improve patient safety and reduce the likelihood of treatment errors.

Implementation of a "No Fly" safety culture in a multicenter radiation medicine department

PURPOSE

The safe delivery of radiation therapy requires multiple disciplines and interactions to perform flawlessly for each patient. Because treatment is individualized and every aspect of the patient's care is unique, it is difficult to regiment a delivery process that works flawlessly. The purpose of this study is to describe one safety-directed component of our quality program called the "No Fly Policy" (NFP).

METHODS AND MATERIALS

Our quality assurance program for radiation therapy reviewed the entire process of care prior, during, and after a patient's treatment course. Each component of care was broken down and rebuilt within a matrix of multidisciplinary safety quality checklists (QCL). The QCL process map was subsequently streamlined with revised task due dates and stopping rules. The NFP was introduced to place a holding pattern on treatment initiation pending reconciliation of associated stopping events. The NFP was introduced in a pilot phase using a Six-Sigma process improvement approach. Quantitative analysis on the performance of the new QCLs was performed using crystal reports in the Oncology Information Systems. Root cause analysis was conducted.

RESULTS

Notable improvements in QCL performance were observed. The variances among staff in completing tasks reduced by a factor of at least 3, suggesting better process control. Steady improvements over time indicated an increasingly compliant and controlled adoption of the new safety-oriented process map. Stopping events led to rescheduling treatments with average and maximum delays of 2 and 4 days, respectively, with no reported adverse effects. The majority of stopping events were due to incomplete plan approvals stemming from treatment planning delays. Whereas these may have previously solicited last-minute interventions, including intensity modulated radiation therapy quality assurance, the NFP enabled nonpunitive, reasonable schedule adjustments to mitigate compromises in safe delivery.

CONCLUSIONS

Implementation of the NFP has helped to mitigate risk from expedited care, convert reactive to proactive delays, and created a checklist, process driven, and variance-reducing culture in a large, multicenter department.

Technological advancements and error rates in radiation therapy delivery

PURPOSE

Technological advances in radiation therapy (RT) delivery have the potential to reduce errors via increased automation and built-in quality assurance (QA) safeguards, yet may also introduce new types of errors. Intensity-modulated RT (IMRT) is an increasingly used technology that is more technically complex than three-dimensional (3D)-conformal RT and conventional RT. We determined the rate of reported errors in RT delivery among IMRT and 3D/conventional RT treatments and characterized the errors associated with the respective techniques to improve existing QA processes.

METHODS AND MATERIALS

All errors in external beam RT delivery were prospectively recorded via a nonpunitive error-reporting system at Brigham & Women's Hospital/Dana Farber Cancer Institute. Errors are defined as any unplanned deviation from the intended RT treatment and are reviewed during monthly departmental quality improvement meetings. We analyzed all reported errors since the routine use of IMRT in our department, from January 2004 to July 2009. Fisher's exact test was used to determine the association between treatment technique (IMRT vs. 3D/conventional) and specific error types. Effect estimates were computed using logistic regression.

RESULTS

There were 155 errors in RT delivery among 241,546 fractions (0.06%), and none were clinically significant. IMRT was commonly associated with errors in machine parameters (nine of 19 errors) and data entry and interpretation (six of 19 errors). IMRT was associated with a lower rate of reported errors compared with 3D/conventional RT (0.03% vs. 0.07%, p = 0.001) and specifically fewer accessory errors (odds ratio, 0.11; 95% confidence interval, 0.01-0.78) and setup errors (odds ratio, 0.24; 95% confidence interval, 0.08-0.79).

CONCLUSIONS

The rate of errors in RT delivery is low. The types of errors differ significantly between IMRT and 3D/conventional RT, suggesting that QA processes must be uniquely adapted for each technique. There was a lower error rate with IMRT compared with 3D/conventional RT, highlighting the need for sustained vigilance against errors common to more traditional treatment techniques.

Radiation Oncology Safety Information System (ROSIS)--profiles of participants and the first 1074 incident reports

BACKGROUND AND PURPOSE

The Radiation Oncology Safety Information System (ROSIS) was established in 2001. The aim of ROSIS is to collate and share information on incidents and near-incidents in radiotherapy, and to learn from these incidents in the context of departmental infrastructure and procedures.

MATERIALS AND METHODS

A voluntary web-based cross-organisational and international reporting and learning system was developed (cf. the www.rosis.info website). Data is collected via online Department Description and Incident Report Forms. A total of 101 departments, and 1074 incident reports are reviewed.

RESULTS

The ROSIS departments represent about 150,000 patients, 343 megavoltage (MV) units, and 114 brachytherapy units. On average, there are 437 patients per MV unit, 281 per radiation oncologist, 387 per physicist and 353 per radiation therapy technologist (RT/RTT). Only 14 departments have a completely networked system of electronic data transfer, while 10 departments have no electronic data transfer. On average seven quality assurance (QA) or quality control (QC) methods are used at each department. A total of 1074 ROSIS reports are analysed; 97.7% relate to external beam radiation treatment and 50% resulted in incorrect irradiation. Many incidents arise during pre-treatment but are not detected until later in the treatment process. Where an incident is not detected prior to treatment, an average of 22% of the prescribed treatment fractions were delivered incorrectly. The most commonly reported detection methods were "found at time of patient treatment" and during "chart-check".

CONCLUSION

While the majority of the incidents that reported to this international cross-organisational reporting system are of minor dosimetric consequence, they affect on average more than 20% of the patient's treatment fractions. Nonetheless, defence-in-depth is apparent in departments registered with ROSIS. This indicates a need for further evaluation of the effectiveness of quality controls.

The use of human factors methods to identify and mitigate safety issues in radiation therapy

BACKGROUND AND PURPOSE

New radiation therapy technologies can enhance the quality of treatment and reduce error. However, the treatment process has become more complex, and radiation dose is not always delivered as intended. Using human factors methods, a radiotherapy treatment delivery process was evaluated, and a redesign was undertaken to determine the effect on system safety.

MATERIAL AND METHODS

An ethnographic field study and workflow analysis was conducted to identify human factors issues of the treatment delivery process. To address specific issues, components of the user interface were redesigned through a user-centered approach. Sixteen radiation therapy students were then used to experimentally evaluate the redesigned system through a usability test to determine the effectiveness in mitigating use errors.

RESULTS

According to findings from the usability test, the redesigned system successfully reduced the error rates of two common errors (p<.04 and p<.01). It also improved the mean task completion time by 5.5% (p<.02) and achieved a higher level of user satisfaction.

CONCLUSIONS

These findings demonstrated the importance and benefits of applying human factors methods in the design of radiation therapy systems. Many other opportunities still exist to improve patient safety in this area using human factors methods.