An Audit for Radiotherapy Planning and Treatment Errors From a Low-Middle-Income Country Centre

Results of an Artificial Intelligence-Based Image Review System to Detect Patient Misalignment Errors in a Multi-institutional Database of Cone Beam Computed Tomography-Guided Radiation Therapy

PURPOSE

Present knowledge of patient setup and alignment errors in image guided radiation therapy (IGRT) relies on voluntary reporting, which is thought to underestimate error frequencies. A manual retrospective patient-setup misalignment error search is infeasible owing to the bulk of cases to be reviewed. We applied a deep learning-based misalignment error detection algorithm (EDA) to perform a fully automated retrospective error search of clinical IGRT databases and determine an absolute gross patient misalignment error rate.

METHODS AND MATERIALS

The EDA was developed to analyze the registration between planning scans and pretreatment cone beam computed tomography scans, outputting a misalignment score ranging from 0 (most unlikely) to 1 (most likely). The algorithm was trained using simulated translational errors on a data set obtained from 680 patients treated at 2 radiation therapy clinics between 2017 and 2022. A receiver operating characteristic analysis was performed to obtain target thresholds. DICOM Query and Retrieval software was integrated with the EDA to interact with the clinical database and fully automate data retrieval and analysis during a retrospective error search from 2016 to 2017 and from 2021 to 2022 for the 2 institutions, respectively. Registrations were flagged for human review using both a hard-thresholding method and a prediction trending analysis over each individual patient's treatment course. Flagged registrations were manually reviewed and categorized as errors (>1 cm misalignment at the target) or nonerrors.

RESULTS

A total of 17,612 registrations were analyzed by the EDA, resulting in 7.7% flagged events. Three previously reported errors were successfully flagged by the EDA, and 4 previously unreported vertebral body misalignment errors were discovered during case reviews. False positive cases often displayed substantial image artifacts, patient rotation, and soft tissue anatomy changes.

CONCLUSIONS

Our results validated the clinical utility of the EDA for bulk image reviews and highlighted the reliability and safety of IGRT, with an absolute gross patient misalignment error rate of 0.04% ± 0.02% per delivered fraction.

Errors in manual radiotherapy treatment procedures and their evolution in a low resource setting: Uganda's experience

PURPOSE

In Uganda, two-dimensional (2D) radiotherapy treatments have been in use since the establishment of radiotherapy in 1995. Preliminary investigations of treatment records in November 2019 showed evidence of gaps requiring urgent attention. The purpose of this study was to improve the safety of the treatments.

METHODS

Records of 1164 patients treated in 1387 courses (1412 sites) on Cobalt-60 units were reviewed todetermine the frequency and dosimetric implications of events that occurred at different stepsof the radiotherapy process. The results were presented and discussed with the differentprofessionals for learning purposes.

RESULTS

Most common dosimetric eventswere omission of block tray, bolus and couch transmission factors in time calculations, incorrect field sizes and depths, wrong beam weighting, independent calculations and prescription doses contributing 28.6 %, 10.1 %, 6.0 %,11.9 %, 10.1 %, 5.4 %, 4.8 % and 8.9 % to the 168 observed errors. Comparison of the calculated treatment doses with the prescribed doses showed that 88 % of the 1412 sites were treated with radiation doses within an accuracy of ± 5 %. However, an analysis of the evolution along the years demonstrated an improvement from 82.8 % in 2018 to 86.1 % in 2019, and 93.2 % in 2020. Most common procedural events were incomplete setup instructions and missing patient data in the record and verify system of the Co-60 units for 57 % and 60.1 % of the 1164 patients.

CONCLUSIONS

Opportunities for improvement of safety in the delivery of radiotherapy treatments were identified. Learning from these past errors should raise awareness in the team leading to a safer treatments.

Quality Improvement Process with Incident Learning Program Helped Reducing Transcriptional Errors on Telecobalt Due to Mismatched Parameters in Different Generations

Purpose

Higher frequency of transcriptional errors in the radiotherapy electronic charts for patients on telecobalt was noted. We describe the impact of the quality improvement (QI) initiative under the department's incident learning program (ILP).

Materials and Methods

The multidisciplinary quality team under ILP was formed to identify the root cause and introduce methods to reduce (smart goal) the current transcription error rate of 40% to <5% over 12 months. A root cause analysis including a fishbone diagram, Pareto chart, and action prioritization matrix was done to identify key drivers and interventions. Plan-Do-Study-Act (PDSA) Cycle strategy was undertaken. The primary outcome was percentage charts with transcriptional errors per month. The balancing measure was "new errors" due to interventions. All errors were identified and corrected before patient treatment.

Results

The average baseline error rate was 44.14%. The two key drivers identified were education of the workforce involved and mechanical synchronization of various machine parameters. PDSA cycle 1 consisted of an education program and sensitization of the staff, post which the error rates dropped to 5.4% (t-test P = 0.03). Post-PDSA cycle 2 (synchronization of machine parameters), 1, 3, and 6 months and 1 year, the error rates were sustained to 5%, 4%, 3%, and 4% (t-test P > 0.05) with no new additional errors.

Conclusions

With various generations of machines and technologies that are not synchronized, the proneness of transcription errors can be very high which can be identified and corrected with a typical QI process under ILP.

Development and inter-institutional validation of an automatic vertebral-body misalignment error detector for Cone-Beam CT guided radiotherapy

BACKGROUND

In Cone-Beam Computed Tomography (CBCT) guided radiotherapy, off-by-one vertebral body misalignments are rare but serious errors which lead to wrong-site treatments.

PURPOSE

An automatic error detection algorithm was developed that uses a three-branch convolutional neural network error detection model (EDM) to detect off-by-one vertebral body misalignments using planning computed tomography (CT) images and setup CBCT images.

METHODS

Algorithm training and test data consisted of planning CTs and CBCTs from 480 patients undergoing radiotherapy treatment in the thoracic and abdominal regions at two radiotherapy clinics. The clinically applied registration was used to derive true-negative (no error) data. The setup and planning images were then misaligned by one vertebral body in both the superior and inferior directions, simulating the most likely misalignment scenarios. For each of the aligned and misaligned 3D image pairs, 2D slice pairs were automatically extracted in each anatomical plane about a point within the vertebral column. The three slice pairs obtained were then inputted to the EDM which returned a probability of vertebral misalignment. One model (EDM₁ ) was trained solely on data from institution #1. EDM₁ was further trained using a lower learning rate on a dataset from institution #2 to produce a fine-tuned model, EDM₂ . Another model, EDM₃ , was trained from scratch using a training dataset composed of data from both institutions. These three models were validated on a randomly selected and unseen dataset composed of images from both institutions, for a total of 303 image pairs. The model performances were quantified using a receiver operating characteristic analysis. Due to the rarity of vertebral body misalignments in the clinic, a minimum threshold value yielding a specificity of at least 99% was selected. Using this threshold, the sensitivity was calculated for each model, on each institution's test set separately.

RESULTS

When applied to the combined test set, EDM₁ , EDM₂ , and EDM₃ resulted in an area under curve of 99.5%, 99.4% and 99.5%, respectively. EDM₁ achieved a sensitivity of 96% and 88% on Institution #1 and Institution #2 test set, respectively. EDM₂ obtained a sensitivity of 95% on each institution's test set. EDM₃ achieved a sensitivity of 95% and 88% on Institution #1 and Institution #2 test set, respectively.

CONCLUSION

The proposed algorithm demonstrated accuracy in identifying off-by-one vertebral body misalignments in CBCT-guided radiotherapy that was sufficiently high to allow for practical implementation. It was found that fine-tuning the model on a multi-facility dataset can further enhance the generalizability of the algorithm. This article is protected by copyright. All rights reserved.

Quality and Safety With Technological Advancements in Radiotherapy: An Overview and Journey Narrative From a Low- and Middle-Income Country Institution

To present an overview of quality and safety in radiotherapy from the context of low- and middle-income countries on the basis of a recently conducted annual meeting of our institution and our experience of implementing an error management system at our center.

Quality and safety improvement with evolving technology in LMIC, a journey described.

Multi-institutional dose audit in radiotherapy facilities using in-house developed optically stimulated luminescence disc dosimeters

Aim

The aim of this study was to carried out the audit of radiotherapy centers practicing conformal radiotherapy techniques and demonstrate the suitability of this indigenous optically stimulated luminescence (OSL) disc dosimeters in beam quality audit and verification of patient-specific dosimetry in conventional and conformal treatments in radiotherapy.

Materials and Methods

Dose audit in conventional and conformal (intensity-modulated radiotherapy and volumetric-modulated arc therapy) radiotherapy techniques was conducted using in-house developed Al2O3:C-based OSL disc dosimeter and commercially available Gafchromic EBT3 film in 6 MV (flat and unflat) photon and 6 and 15 MeV electron beams. OSL disc dosimeter and Gafchromic EBT3 film measured dose values were verified using the ionization chamber measurements.

Results

Percentage variations of doses measured by OSL disc dosimeters and EBT3 Gafchromic film for conventional radiotherapy technique were in the range of 0.15%-4.6% and 0.40%-5.45%, respectively, with respect to the treatment planning system calculated dose values. For conformal radiotherapy techniques, the percentage variations of OSL disc and EBT3 film measured doses were in the range of 0.1%-4.9% and 0.3%-5.0%, respectively.

Conclusion

The results of this study supported by statistical evidence provided the confidence that indigenously developed Al2O3:C-based OSL disc dosimeters are suitable for dose audit in conventional and advanced radiotherapy techniques.

Use of a Checklist Approach on a Telecobalt in an Attempt to Reduce Human Errors in Radiotherapy Delivery and Improve Therapeutic Ratio

Background:

The process of radiotherapy treatment planning and delivery involves multiple steps and professionals causing it to be prone to errors. Radiotherapy centers equipped with old telecobalt machines have certain peculiar challenges to workflow. We designed and tested a checklist for radiotherapy technicians (RTTs) to reduce chances of error during treatment delivery on a telecobalt machine.

Materials and Methods:

A physical checklist was designed for RTTs to use in the pretreatment pause using a template advocated by the American Association of Physicists in Medicine. It was tested on 4 RTTs over 1000 radiotherapy delivery sittings.

Results:

The checklist helped to rectify 41 documentary lapses and 28 errors in radiotherapy treatment parameters while also identifying 12 instances where treatment plan modifications were due and 30 where the patient was due for review by the radiation oncologist. The average time to go through the checklist was between 2.5 and 3 min.

Conclusions:

The development and use of the checklist has helped in reducing errors and also improving workflow in our department. It is recommended to utilize such physical checklists in all radiotherapy centers with telecobalt machines. The success of the checklist depends upon leadership, teamwork, acceptance of a need to inculcate a “safety culture,” with voluntary error-reporting and a willingness to learn from such errors.