The Impact of Transitioning to Prospective Contouring and Planning Rounds as Peer Review

Implementation of a Novel Chart Rounds Application to Facilitate Peer Review in a Virtual Academic Environment

Purpose

Peer review in the form of chart rounds is a critical component of quality assurance and safety in radiation therapy treatments. Radiation therapy departments have undergone significant changes that impose challenges to meaningful review, including institutional growth and increasing use of virtual environment. We discuss the implementation of a novel chart rounds (NCR) format and application adapted to modern peer review needs at a single high-volume multisite National Cancer Institute designated cancer center.

Methods and Materials

A working group was created to improve upon the prior institutional chart rounds format (standard chart rounds or SCR). Using a novel in-house application and format redesign, an NCR was created and implemented to accomplish stated goals. Data regarding the SCR and NCR system were then extracted for review.

Results

SCR consisted of 2- 90-minute weekly sessions held to review plans across all disease sites, review of 49 plans per hour on average. NCR uses 1-hour long sessions divided by disease site, enabling additional time to be spent per patient (11 plans per hour on average) and more robust discussion. The NCR application is able to automate a list of plans requiring peer review from the institutional treatment planning system. The novel application incorporates features that enable efficient and accurate review of plans in the virtual setting across multiple sites. A systematic scoring system is integrated into the application to record feedback. Over 5 months of use of the NCR, 1160 plans have been reviewed with 143 scored as requiring minor changes, 32 requiring major changes and 307 with comments. Major changes triggered treatment replan. Feedback from scoring is incorporated into physician workflow to ensure changes are addressed.

Conclusion

The presented NCR format and application enables standardized and highly reliable peer review of radiation therapy plans that is robust across a variety of complex planning scenarios and could be implemented globally.

Radiation Oncology Peer Review in a Community Setting: The Value of Prospective Review

Peer review is an important component of any radiation oncology continuous quality improvement program. While limited guidelines exist, there is no consensus about how peer review should be performed, and large variations exist among different institutions. The purpose of this report is to describe our experience with peer review at a busy Radiation Oncology clinic and to evaluate the difference between prospective and retrospective peer review. We also performed a failure modes and effects analysis (FMEA) of the peer review process. Starting in 2015, every peer review session was tracked, including recommended changes to treatment plans. We reviewed the frequency, types and severity of these changes. A team of physicians and physicists conducted an FMEA of the peer review process. Between April 2015 and June 2020, a total of 3,691 patients were peer-reviewed. Out of those, 1,903 were prospective reviews (51.6%). Plans reviewed before treatment were almost 4.5 times more likely to be changed by peer review than those reviewed after the start of treatment (0.9% vs 0.2%). Plan changes after the start of treatment had a higher severity than changes prior to the start of treatment. FMEA identified several critical components of peer review. While there is no national standard for peer review, it is evident that prospective peer review is preferable. There may be a subconscious reluctance to change plans already underway, which could be a barrier to improving plans with the peer review process. Rather than reviewing in a group setting, it would be ideal to individually assign review tasks that are embedded in the clinical flow, assuring prospective review for all patients prior to final physician approval. Individual review rather than group review may be more candid, due to interpersonal concerns about publicly disagreeing with colleagues.

Analysis of Retrospective Versus Prospective Peer Review in a Multisite Academic Radiation Department

Purpose

Our multisite academic radiation department reviewed our experience with transitioning from weekly primarily retrospective to daily primarily prospective peer review to improve plan quality and decrease the rate of plan revisions after treatment start.

Methods and Materials

This study was an institutional review board-approved prospective comparison of radiation treatment plan review outcomes of plans reviewed weekly (majority within 1 week after treatment start) versus plans reviewed daily (majority before treatment start, except brachytherapy, frame-based radiosurgery, and some emergent plans). Deviations were based on peer comments and considered major if plan revisions were recommended before the next fraction and minor if modifications were suggested but not required. Categorical variables were compared using χ2 distribution tests of independence; means were compared using independent t tests.

Results

In all, 798 patients with 1124 plans were reviewed: 611 plans weekly and 513 plans daily. Overall, 76 deviations (6.8%) were noted. Rates of any deviation were increased in the daily era (8.6% vs 5.2%; P = .026), with higher rates of major deviations in the daily era (4.1% vs 1.6%; P = .012). Median working days between initial simulation and treatment was the same across eras (8 days). Deviations led to a plan revision at a higher rate in the daily era (84.1% vs 31.3%; P < .001).

Conclusions

Daily prospective peer review is feasible in a multisite academic setting. Daily peer review with emphasis on prospective plan evaluation increased constructive plan feedback, plan revisions, and plan revisions being implemented before treatment start.

Improving Quality Metrics in Radiation Oncology: Implementation of Pretreatment Peer Review for Stereotactic Body Radiation Therapy in Patients with Thoracic Cancer

Purpose

Traditional peer reviews occur weekly, and can take place up to 1 week after the start of treatment. The American Society for Radiation Oncology peer-review white paper identified stereotactic body radiation therapy (SBRT) as a high priority for contour/plan review before the start of treatment, considering both the rapid-dose falloff and short treatment course. Yet, peer-review goals for SBRT must also balance physician time demands and the desire to avoid routine treatment delays that would occur in the setting of a 100% pretreatment (pre-Tx) review compliance requirement or prolonging the standard treatment planning timeline. Herein, we report on our pilot experience of a pre-Tx peer review of thoracic SBRT cases.

Methods and Materials

From March 2020 to August 2021, patients undergoing thoracic SBRT were identified for pre-Tx review, and placed on a quality checklist. We implemented twice-weekly meetings for detailed pre-Tx review of organ-at-risk/target contours and dose constraints in the treatment planning system for SBRT cases. Our quality metric goal was to peer review ≥90% of SBRT cases before exceeding 25% of the dose delivered. We used a statistical process control chart with sigma limits (ie, standard deviations [SDs]) to access compliance rates with pre-Tx review implementation.

Results

We identified 252 patients treated with SBRT to 294 lung nodules. When comparing pre-Tx review completion from initial rollout to full implementation, our rates improved from 19% to 79% (ie, from 1 sigma limit [SDs]) below to >2 sigma limits (SDs) above. Additionally, early completion of any form of contour/plan review (defined as any pre-Tx or standard review completed before exceeding 25% of the dose delivered) increased from 67% to 85% (March 2020-November 2020) to 76% to 94% (December 2020-August 2021).

Conclusions

We successfully implemented a sustainable workflow for detailed pre-Tx contour/plan review for thoracic SBRT cases in the context of twice-weekly disease site-specific peer-review meetings. We reached our quality improvement objective to peer review ≥90% of SBRT cases before exceeding 25% of the dose delivered. This process was feasible to conduct in an integrated network of sites across our system.

Physician review of image registration and normal structure delineation

Introduction

Methods

Results

Conclusion

100% peer review in radiation oncology: is it feasible?

Purpose

Peer review has been proposed as a strategy to ensure patient safety and plan quality in radiation oncology. Despite its potential benefits, barriers commonly exist to its optimal implementation in daily clinical routine. Our purpose is to analyze peer-review process at our institution.

Methods and materials

Based on our group peer-review process, we quantified the rate of plan changes, time and resources needed for this process. Prospectively, data on cases presented at our institutional peer-review conference attended by physicians, resident physicians and physicists were collected. Items such as time to present per case, type of patient (adult or pediatric), treatment intent, dose, aimed technique, disease location and receipt of previous radiation were gathered. Cases were then analyzed to determine the rate of major change, minor change and plan rejection after presentation as well as the median time per session.

Results

Over a period of 4 weeks, 148 cases were reviewed. Median of attendants was six physicians, three in-training-physicians and one physicist. Median time per session was 38 (4–72) minutes. 59.5% of cases presented in 1–4 min, 32.4% in 5–9 min and 8.1% in ≥ 10 min. 79.1% of cases were accepted without changes, 11.5% with minor changes, 6% with major changes and 3.4% were rejected with indication of new presentation. Most frequent reason of change was contouring corrections (53.8%) followed by dose or fractionation (26.9%).

Conclusion

Everyday group consensus peer review is an efficient manner to recollect clinical and technical data of cases presented to ensure quality radiation care before initiation of treatment as well as ensuring department quality in a feedback team environment. This model is feasible within the normal operation of every radiation oncology Department.

The Fusion of Incident Learning and Failure Mode and Effects Analysis for Data-Driven Patient Safety Improvements

PURPOSE

Incident learning is a critical part of the quality improvement process for all radiation therapy clinics. Failure mode and effects analysis has also been adopted as a hazard analysis method within the field of radiation oncology based on the recommendations of American Association of Physicists in Medicine Task Group 100. In this work, we demonstrate a fusion of these techniques that is efficient and transferrable to all types of clinics and that allows data-driven targeting of the highest risk error types.

METHODS AND MATERIALS

Four clinical physicists recorded safety events detected during physics treatment plan quality assurance over a 27-month period. Events were sorted into the broad categories of either a documentation or plan construction error. Events were further stratified into subcategories until sufficiently discriminated against for analysis. Event risks were quantified using reduced-resolution TG-100 severity scores combined with observed occurrence rates. The highest risk categories were examined for intervention strategies.

RESULTS

A total of 871 events were identified over the study period. Of these, 652 (74.9%) were classified as low severity, 178 (20.4%) as medium severity, and 41 (4.7%) as high severity. Four of the top 5 ranked categories could be targeted by a preplanning chart rounds. Several of the categories could be targeted by additional automation in the planning and QA processes.

CONCLUSIONS

The retrospective classification and risk analysis of safety events allows clinics to design targeted workflow and quality assurance changes aimed at reducing the occurrence of high-risk events. The method presented here leverages incident learning efforts that many clinics are already performing, allows the severity of events to be efficiently assigned, and generates actionable results without requiring a complete failure mode and effects analysis.