Quality management in radiation therapy: A 15 year review of incident reporting in two integrated cancer centres

Results of an Artificial Intelligence-Based Image Review System to Detect Patient Misalignment Errors in a Multi-institutional Database of Cone Beam Computed Tomography-Guided Radiation Therapy

PURPOSE

Present knowledge of patient setup and alignment errors in image guided radiation therapy (IGRT) relies on voluntary reporting, which is thought to underestimate error frequencies. A manual retrospective patient-setup misalignment error search is infeasible owing to the bulk of cases to be reviewed. We applied a deep learning-based misalignment error detection algorithm (EDA) to perform a fully automated retrospective error search of clinical IGRT databases and determine an absolute gross patient misalignment error rate.

METHODS AND MATERIALS

The EDA was developed to analyze the registration between planning scans and pretreatment cone beam computed tomography scans, outputting a misalignment score ranging from 0 (most unlikely) to 1 (most likely). The algorithm was trained using simulated translational errors on a data set obtained from 680 patients treated at 2 radiation therapy clinics between 2017 and 2022. A receiver operating characteristic analysis was performed to obtain target thresholds. DICOM Query and Retrieval software was integrated with the EDA to interact with the clinical database and fully automate data retrieval and analysis during a retrospective error search from 2016 to 2017 and from 2021 to 2022 for the 2 institutions, respectively. Registrations were flagged for human review using both a hard-thresholding method and a prediction trending analysis over each individual patient's treatment course. Flagged registrations were manually reviewed and categorized as errors (>1 cm misalignment at the target) or nonerrors.

RESULTS

A total of 17,612 registrations were analyzed by the EDA, resulting in 7.7% flagged events. Three previously reported errors were successfully flagged by the EDA, and 4 previously unreported vertebral body misalignment errors were discovered during case reviews. False positive cases often displayed substantial image artifacts, patient rotation, and soft tissue anatomy changes.

CONCLUSIONS

Our results validated the clinical utility of the EDA for bulk image reviews and highlighted the reliability and safety of IGRT, with an absolute gross patient misalignment error rate of 0.04% ± 0.02% per delivered fraction.

Improving the safety of radiotherapy treatment processes via incident-driven FMEA feedback loops

BACKGROUND

Failure mode and effects analysis (FMEA) is a valuable tool for radiotherapy risk assessment, yet its outputs might be unreliable due to failures not being identified or due to a lack of accurate error rates.

PURPOSE

A novel incident reporting system (IRS) linked to an FMEA database was tested and evaluated. The study investigated whether the system was suitable for validating a previously performed analysis and whether it could provide accurate error rates to support the expert occurrence ratings of previously identified failure modes.

METHODS

Twenty-three pre-identified failure modes of our external beam radiotherapy process, covering the process steps from patient admission to treatment delivery, were proffered on dedicated FMEA feedback and incident reporting terminals generated by the IRS. The clinical setting involved a computed tomography scanner, dosimetry, and five linacs. Incoming reports were used as basis to identify additional failure modes or confirm initial ones. The Kruskal-Wallis H test was applied to compare the risk priorities of the retrospective and prospective failure modes. Wald's sequential probability ratio test was used to investigate the correctness of the experts' occurrence ratings by means of the number of incoming reports.

RESULTS

Over a 15-month period, 304 reports were submitted. There were 0.005 (confidence interval [CI], 0.0014-0.0082) reported incidents per imaging study and 0.0006 (CI, 0.0003-0.0009) reported incidents per treatment fraction. Sixteen additional failure modes could be identified, and their risk priorities did not differ from those of the initial failure modes (p = 0.954). One failure mode occurrence rating could be increased, whereas the other 22 occurrence ratings could not be disproved.

CONCLUSIONS

Our approach is suitable for validating FMEAs and deducing additional failure modes on a continual basis. Accurate error rates can only be provided if a sufficient number of reports is available.

Automated detection of vertebral body misalignments in orthogonal kV and MV guided radiotherapy: application to a comprehensive retrospective dataset

Objective. In image-guided radiotherapy (IGRT), off-by-one vertebral body misalignments are rare but potentially catastrophic. In this study, a novel detection method for such misalignments in IGRT was investigated using densely-connected convolutional networks (DenseNets) for applications towards real-time error prevention and retrospective error auditing.Approach. A total of 4213 images acquired from 527 radiotherapy patients aligned with planar kV or MV radiographs were used to develop and test error-detection software modules. Digitally reconstructed radiographs (DRRs) and setup images were retrieved and co-registered according to the clinically applied alignment contained in the DICOM REG files. A semi-automated algorithm was developed to simulate patient positioning errors on the anterior-posterior (AP) and lateral (LAT) images shifted by one vertebral body. A DenseNet architecture was designed to classify either AP images individually or AP and LAT image pairs. Receiver-operator characteristic curves (ROC) and areas under the curves (AUC) were computed to evaluate the classifiers on test subsets. Subsequently, the algorithm was applied to the entire dataset in order to retrospectively determine the absolute off-by-one vertebral body error rate for planar radiograph guided RT at our institution from 2011-2021.Main results. The AUCs for the kV models were 0.98 for unpaired AP and 0.99 for paired AP-LAT. The AUC for the MV AP model was 0.92. For a specificity of 95%, the paired kV model achieved a sensitivity of 99%. Application of the model to the entire dataset yielded a per-fraction off-by-one vertebral body error rate of 0.044% [0.0022%, 0.21%] for paired kV IGRT including one previously unreported error.Significance. Our error detection algorithm was successful in classifying vertebral body positioning errors with sufficient accuracy for retrospective quality control and real-time error prevention. The reported positioning error rate for planar radiograph IGRT is unique in being determined independently of an error reporting system.

Risk management patterns in radiation oncology-results of a national survey within the framework of the Patient Safety in German Radiation Oncology (PaSaGeRO) project

PURPOSE

Risk management (RM) is a key component of patient safety in radiation oncology (RO). We investigated current approaches on RM in German RO within the framework of the Patient Safety in German Radiation Oncology (PaSaGeRO) project. Aim was not only to evaluate a status quo of RM purposes but furthermore to discover challenges for sustainable RM that should be addressed in future research and recommendations.

METHODS

An online survey was conducted from June to August 2021, consisting of 18 items on prospective and reactive RM, protagonists of RM, and self-assessment concerning RM. The survey was designed using LimeSurvey and invitations were sent by e‑mail. Answers were requested once per institution.

RESULTS

In all, 48 completed questionnaires from university hospitals, general and non-academic hospitals, and private practices were received and considered for evaluation. Prospective and reactive RM was commonly conducted within interprofessional teams; 88% of all institutions performed prospective risk analyses. Most institutions (71%) reported incidents or near-events using multiple reporting systems. Results were presented to the team in 71% for prospective analyses and 85% for analyses of incidents. Risk conferences take place in 46% of institutions. 42% nominated a manager/committee for RM. Knowledge concerning RM was mostly rated "satisfying" (44%). However, 65% of all institutions require more information about RM by professional societies.

CONCLUSION

Our results revealed heterogeneous patterns of RM in RO departments, although most departments adhered to common recommendations. Identified mismatches between recommendations and implementation of RM provide baseline data for future research and support definition of teaching content.

Development and inter-institutional validation of an automatic vertebral-body misalignment error detector for Cone-Beam CT guided radiotherapy

BACKGROUND

In Cone-Beam Computed Tomography (CBCT) guided radiotherapy, off-by-one vertebral body misalignments are rare but serious errors which lead to wrong-site treatments.

PURPOSE

An automatic error detection algorithm was developed that uses a three-branch convolutional neural network error detection model (EDM) to detect off-by-one vertebral body misalignments using planning computed tomography (CT) images and setup CBCT images.

METHODS

Algorithm training and test data consisted of planning CTs and CBCTs from 480 patients undergoing radiotherapy treatment in the thoracic and abdominal regions at two radiotherapy clinics. The clinically applied registration was used to derive true-negative (no error) data. The setup and planning images were then misaligned by one vertebral body in both the superior and inferior directions, simulating the most likely misalignment scenarios. For each of the aligned and misaligned 3D image pairs, 2D slice pairs were automatically extracted in each anatomical plane about a point within the vertebral column. The three slice pairs obtained were then inputted to the EDM which returned a probability of vertebral misalignment. One model (EDM₁ ) was trained solely on data from institution #1. EDM₁ was further trained using a lower learning rate on a dataset from institution #2 to produce a fine-tuned model, EDM₂ . Another model, EDM₃ , was trained from scratch using a training dataset composed of data from both institutions. These three models were validated on a randomly selected and unseen dataset composed of images from both institutions, for a total of 303 image pairs. The model performances were quantified using a receiver operating characteristic analysis. Due to the rarity of vertebral body misalignments in the clinic, a minimum threshold value yielding a specificity of at least 99% was selected. Using this threshold, the sensitivity was calculated for each model, on each institution's test set separately.

RESULTS

When applied to the combined test set, EDM₁ , EDM₂ , and EDM₃ resulted in an area under curve of 99.5%, 99.4% and 99.5%, respectively. EDM₁ achieved a sensitivity of 96% and 88% on Institution #1 and Institution #2 test set, respectively. EDM₂ obtained a sensitivity of 95% on each institution's test set. EDM₃ achieved a sensitivity of 95% and 88% on Institution #1 and Institution #2 test set, respectively.

CONCLUSION

The proposed algorithm demonstrated accuracy in identifying off-by-one vertebral body misalignments in CBCT-guided radiotherapy that was sufficiently high to allow for practical implementation. It was found that fine-tuning the model on a multi-facility dataset can further enhance the generalizability of the algorithm. This article is protected by copyright. All rights reserved.

Webinar and survey on quality management principles within the Australian and New Zealand ACPSEM Workforce

Healthcare relies upon the accurate and safe delivery of patient care. This is only achievable when systems are developed to ensure high quality, robust outcomes, for instance quality management systems. The concept of quality management can take on a different meaning depending on the context in which it is found. To add complication, the amount of education required for quality management will vary depending on one's exposure to the implementation of quality systems. In part to address these issues, the Australasian College of Physical Scientists and Engineers in Medicine (ACPSEM) Queensland Branch held a quality management webinar for members and non-members across Australia and New Zealand. The purpose of the webinar was to educate and facilitate discussion regarding the application of quality management principles for the ACPSEM profession. In conjunction, a pre- and post-webinar survey was conducted to gain an insight into existing knowledge and attitudes within the professions governed by the ACPSEM and students undertaking related studies. This paper authored by the webinar speakers reintroduces the quality management principles that were discussed in webinar, exemplifies the importance of quality management skills within the ACPSEM professions and presents the results of the surveys, promoting the need for more educational resources on quality management tools.

Evaluating incident learning systems and safety culture in two radiation oncology departments

INTRODUCTION

Radiation oncology patient pathways are complex. This complexity creates risk and potential for error to occur. Comprehensive safety and quality management programmes have been developed alongside the use of incident learning systems (ILSs) to mitigate risks and errors reaching patients. Robust ILSs rely on the safety culture (SC) within a department. The aim of this study was to assess perceptions and understanding of SC and ILSs in two closely linked radiation oncology departments and to use the results to consider possible quality improvement (QI) of department ILSs and SC.

METHODS

A survey to assess perceptions of SC and the currently used ILSs was distributed to radiation oncologists, radiation therapists and radiation oncology medical physicists in the two departments. The responses of 95 staff were evaluated (63% of staff). The findings were used to determine any areas for improvement in SC and local ILSs.

RESULTS

Differences were shown between the professional cohorts. Barriers to current ILS use were indicated by 67% of respondents. Positive SC was shown in each area assessed: 69% indicated the departments practised a no-blame culture. Barriers identified in one department prompted a QI project to develop a new reporting system and process, improve departmental learning and modify the overall ILS.

CONCLUSION

An understanding of SC and attitudes to ILSs has been established and used to improve ILS reporting, feedback on incidents, departmental learning and the QA program. This can be used for future comparisons as the systems develop.

Impact of technological and departmental changes on incident rates in radiation oncology over a seventeen-year period

INTRODUCTION

Advancements in technology and processes are designed to bring improvement. However, this is often achieved in parallel with increases in complexity, simultaneously presenting opportunities for new types of errors. This study aims to contextualise the impact of internal departmental changes upon radiation incidents and near misses recorded.

METHODS

A timeline of events and a comprehensive incident categorisation system were applied to all radiation incidents and near misses recorded at the Princess Alexandra Hospital Radiation Oncology department from 2003 to 2019, inclusive. Descriptive statistics were performed to identify the type and number of incidents reported during the time period in relation to potential changes within the department, with a focus on the implementation of an electronic environment.

RESULTS

Over the seventeen-year period, 157 incidents and 76 near misses were reported. The majority of incidents were classified as 'procedural' (78%), with 'treatment' being both the highest point of error and point of detection (49% and 85%, respectively). The largest number of incidents and near misses were reported in 2018 (n = 39) which was also a year that experienced the largest number of departmental changes (n = 16), including the move to a completely electronic planning process.

CONCLUSIONS

Changes within the department were followed by an increasing number of reported incidents. Proactive measures should be undertaken prior to the implementation of major changes within the department to aid in the minimisation of incident occurrence.