A Prospective Analysis of Radiation Oncologist Compliance With Early Peer Review Recommendations

Implementation of a Novel Chart Rounds Application to Facilitate Peer Review in a Virtual Academic Environment

Purpose

Peer review in the form of chart rounds is a critical component of quality assurance and safety in radiation therapy treatments. Radiation therapy departments have undergone significant changes that impose challenges to meaningful review, including institutional growth and increasing use of virtual environment. We discuss the implementation of a novel chart rounds (NCR) format and application adapted to modern peer review needs at a single high-volume multisite National Cancer Institute designated cancer center.

Methods and Materials

A working group was created to improve upon the prior institutional chart rounds format (standard chart rounds or SCR). Using a novel in-house application and format redesign, an NCR was created and implemented to accomplish stated goals. Data regarding the SCR and NCR system were then extracted for review.

Results

SCR consisted of 2- 90-minute weekly sessions held to review plans across all disease sites, review of 49 plans per hour on average. NCR uses 1-hour long sessions divided by disease site, enabling additional time to be spent per patient (11 plans per hour on average) and more robust discussion. The NCR application is able to automate a list of plans requiring peer review from the institutional treatment planning system. The novel application incorporates features that enable efficient and accurate review of plans in the virtual setting across multiple sites. A systematic scoring system is integrated into the application to record feedback. Over 5 months of use of the NCR, 1160 plans have been reviewed with 143 scored as requiring minor changes, 32 requiring major changes and 307 with comments. Major changes triggered treatment replan. Feedback from scoring is incorporated into physician workflow to ensure changes are addressed.

Conclusion

The presented NCR format and application enables standardized and highly reliable peer review of radiation therapy plans that is robust across a variety of complex planning scenarios and could be implemented globally.

Radiation Oncology Peer Review in a Community Setting: The Value of Prospective Review

Peer review is an important component of any radiation oncology continuous quality improvement program. While limited guidelines exist, there is no consensus about how peer review should be performed, and large variations exist among different institutions. The purpose of this report is to describe our experience with peer review at a busy Radiation Oncology clinic and to evaluate the difference between prospective and retrospective peer review. We also performed a failure modes and effects analysis (FMEA) of the peer review process. Starting in 2015, every peer review session was tracked, including recommended changes to treatment plans. We reviewed the frequency, types and severity of these changes. A team of physicians and physicists conducted an FMEA of the peer review process. Between April 2015 and June 2020, a total of 3,691 patients were peer-reviewed. Out of those, 1,903 were prospective reviews (51.6%). Plans reviewed before treatment were almost 4.5 times more likely to be changed by peer review than those reviewed after the start of treatment (0.9% vs 0.2%). Plan changes after the start of treatment had a higher severity than changes prior to the start of treatment. FMEA identified several critical components of peer review. While there is no national standard for peer review, it is evident that prospective peer review is preferable. There may be a subconscious reluctance to change plans already underway, which could be a barrier to improving plans with the peer review process. Rather than reviewing in a group setting, it would be ideal to individually assign review tasks that are embedded in the clinical flow, assuring prospective review for all patients prior to final physician approval. Individual review rather than group review may be more candid, due to interpersonal concerns about publicly disagreeing with colleagues.

Analysis of Retrospective Versus Prospective Peer Review in a Multisite Academic Radiation Department

Purpose

Our multisite academic radiation department reviewed our experience with transitioning from weekly primarily retrospective to daily primarily prospective peer review to improve plan quality and decrease the rate of plan revisions after treatment start.

Methods and Materials

This study was an institutional review board-approved prospective comparison of radiation treatment plan review outcomes of plans reviewed weekly (majority within 1 week after treatment start) versus plans reviewed daily (majority before treatment start, except brachytherapy, frame-based radiosurgery, and some emergent plans). Deviations were based on peer comments and considered major if plan revisions were recommended before the next fraction and minor if modifications were suggested but not required. Categorical variables were compared using χ2 distribution tests of independence; means were compared using independent t tests.

Results

In all, 798 patients with 1124 plans were reviewed: 611 plans weekly and 513 plans daily. Overall, 76 deviations (6.8%) were noted. Rates of any deviation were increased in the daily era (8.6% vs 5.2%; P = .026), with higher rates of major deviations in the daily era (4.1% vs 1.6%; P = .012). Median working days between initial simulation and treatment was the same across eras (8 days). Deviations led to a plan revision at a higher rate in the daily era (84.1% vs 31.3%; P < .001).

Conclusions

Daily prospective peer review is feasible in a multisite academic setting. Daily peer review with emphasis on prospective plan evaluation increased constructive plan feedback, plan revisions, and plan revisions being implemented before treatment start.

Improving Quality Metrics in Radiation Oncology: Implementation of Pretreatment Peer Review for Stereotactic Body Radiation Therapy in Patients with Thoracic Cancer

Purpose

Traditional peer reviews occur weekly, and can take place up to 1 week after the start of treatment. The American Society for Radiation Oncology peer-review white paper identified stereotactic body radiation therapy (SBRT) as a high priority for contour/plan review before the start of treatment, considering both the rapid-dose falloff and short treatment course. Yet, peer-review goals for SBRT must also balance physician time demands and the desire to avoid routine treatment delays that would occur in the setting of a 100% pretreatment (pre-Tx) review compliance requirement or prolonging the standard treatment planning timeline. Herein, we report on our pilot experience of a pre-Tx peer review of thoracic SBRT cases.

Methods and Materials

From March 2020 to August 2021, patients undergoing thoracic SBRT were identified for pre-Tx review, and placed on a quality checklist. We implemented twice-weekly meetings for detailed pre-Tx review of organ-at-risk/target contours and dose constraints in the treatment planning system for SBRT cases. Our quality metric goal was to peer review ≥90% of SBRT cases before exceeding 25% of the dose delivered. We used a statistical process control chart with sigma limits (ie, standard deviations [SDs]) to access compliance rates with pre-Tx review implementation.

Results

We identified 252 patients treated with SBRT to 294 lung nodules. When comparing pre-Tx review completion from initial rollout to full implementation, our rates improved from 19% to 79% (ie, from 1 sigma limit [SDs]) below to >2 sigma limits (SDs) above. Additionally, early completion of any form of contour/plan review (defined as any pre-Tx or standard review completed before exceeding 25% of the dose delivered) increased from 67% to 85% (March 2020-November 2020) to 76% to 94% (December 2020-August 2021).

Conclusions

We successfully implemented a sustainable workflow for detailed pre-Tx contour/plan review for thoracic SBRT cases in the context of twice-weekly disease site-specific peer-review meetings. We reached our quality improvement objective to peer review ≥90% of SBRT cases before exceeding 25% of the dose delivered. This process was feasible to conduct in an integrated network of sites across our system.

Attitudes and access to resources and strategies to improve quality of radiotherapy among US radiation oncologists: A mixed methods study

INTRODUCTION

We aimed to assess contouring-related practices among US radiation oncologists and explore how access to and use of resources and quality improvement strategies vary based on individual- and organization-level factors.

METHODS

We conducted a mixed methods study with a sequential explanatory design. Surveys were emailed to a random 10% sample of practicing US radiation oncologists. Participating physicians were invited to a semi-structured interview. Kruskal-Wallis and Wilcoxon rank-sum tests and a multivariable regression model were used to evaluate associations. Interview data were coded using thematic content analysis.

RESULTS

Survey overall response rate was 24%, and subsequent completion rate was 97%. Contouring-related questions arise in ≥50% of clinical cases among 73% of respondents. Resources accessed first include published atlases (75%) followed by consulting another radiation oncologist (60%). Generalists access consensus guidelines more often than disease-site specialists (P = 0.04), while eContour.org is more often used by generalists (OR 4.3, 95% CI 1.2-14.8) and younger physicians (OR 1.33 for each 5-year increase, 95% CI 1.08-1.67). Common physician-reported barriers to optimizing contour quality are time constraints (58%) and lack of access to disease-site specialists (21%). Forty percent (40%, n = 14) of physicians without access to disease-site specialists indicated it could facilitate the adoption of new treatments. Almost all (97%) respondents have formal peer review, but only 43% have contour-specific review, which is more common in academic centres (P = 0.02).

CONCLUSION

Potential opportunities to improve radiation contour quality include improved access to disease-site specialists and contour-specific peer review. Physician time must be considered when designing new strategies.

Peer review quality assurance in stereotactic body radiotherapy planning: the impact of case volume

Purpose:

Peer review is an essential quality assurance component of radiation therapy planning. A growing body of literature has demonstrated substantial rates of suggested plan changes resulting from peer review. There remains a paucity of data on the impact of peer review rounds for stereotactic body radiation therapy (SBRT). We therefore aim to evaluate the outcomes of peer review in this specific patient cohort.

Methods and materials:

We conducted a retrospective review of all SBRT cases that underwent peer review from July 2015 to June 2018 at a single institution. Weekly peer review rounds are grouped according to cancer subsite and attended by radiation oncologists, medical physicists and medical radiation technologists. We prospectively compiled ‘learning moments’, defined as cases with suggested changes or where an educational discussion occurred beyond routine management, and critical errors, defined as errors which could alter clinical outcomes, recorded prospectively during peer review. Plan changes implemented after peer review were documented.

Results:

Nine hundred thirty-four SBRT cases were included. The most common treatment sites were lung (518, 55%), liver (196, 21%) and spine (119, 13%). Learning moments were identified in 161 cases (17%) and translated into plan changes in 28 cases (3%). Two critical errors (0.2%) were identified: an inadequate planning target volume margin and an incorrect image set used for contouring. There was a statistically significantly higher rate of learning moments for lower-volume SBRT sites (defined as ≤30 cases/year) versus higher-volume SBRT sites (29% vs 16%, respectively; p = 0.001).

Conclusions:

Peer review for SBRT cases revealed a low rate of critical errors, but did result in implemented plan changes in 3% of cases, and either educational discussion or suggestions of plan changes in 17% of cases. All SBRT sites appear to benefit from peer review, though lower-volume sites may require particular attention.

Radiation oncology peer review in Australia and New Zealand

Peer review is a part of high quality care within radiation oncology, designed to achieve the best outcomes for patients. We discuss the importance of and evidence for peer review in clinical practice. The Royal Australia and New Zealand College of Radiologists (RANZCR) has evolved a Peer Review Assessment Tool (PRAT) since 1999. We report the results of a RANZCR faculty survey conducted in radiation oncology facilities across Australia and New Zealand to guide the 2019 PRAT revision process, and discuss the development and implementation of the 2019 PRAT. Peer-review processes are now mandated as a component of Australian and International Quality Standards. Several practical recommendations might address challenges for effective implementation of peer review process in routine clinical practice. This includes prioritising tumour sites and treatment techniques for peer review within the time and resources constraints of each institution, improving resource allocation, ensuring optimal timing and duration for peer review meetings, and adopting multi-centre virtual peer review meeting where necessary.

Evaluation of a prospective radiation oncology departmental team review process using standardized simulation directives

INTRODUCTION

The primary objective of this study is to evaluate the utility and value of an institutional, multi-disciplinary radiation oncology team review process prior to radiotherapy (RT) simulation.

METHODS

Over a period of 3 months and through an iterative team-based process, a standardized simulation requisition directive (SSRD) was developed, piloted, modified, and subsequently implemented for all patients treated with external beam RT at a single tertiary care institution from January to December 2020. The SSRDs were reviewed at a daily multi-disciplinary radiation oncology team review conference; modifications consequential to the review were prospectively recorded in a quality database.

RESULTS

1,500 consecutive SSRDs were prospectively reviewed for this study. 397 modifications on 290 (19.3%) SSRDs were recorded and parsed into 5 main categories and 18 subcategories. The most common modifications resulted from changes in immobilization device (n=88, 22.2%), RT care path (n=56, 14.1%), and arm positioning (n=43, 10.8%). On univariate analysis, modifications were associated with RT intent, scan parameters, tumor site, and consultation type. An increased rate modifications was observed for patients had telemedicine consults (n=101, 22.7%) compared to in-person consultations (n=189, 17.9%) (p=0.032). Using logistic regression analysis, there was also a statistically significant relationship between postoperative RT delivery and modification rates (OR: 2.913, 95% CI: 1.014-8.372) (p=0.0126). Overall, only 14 patients (0.9%) needed re-simulation during the entire study period.

CONCLUSIONS

Prospective multi-disciplinary radiation oncology team review prior to simulation identifies actionable change in approximately 19% of procedures, and results in an extremely low rate (<1%) of re-simulation. As departmental processes transition to virtual platforms, thorough attention is needed to identify patients at higher risk of simulation modifications.

The impact of radiation therapist-led structured peer review meetings on compliance to Radiation Oncology Practice Standards

INTRODUCTION

Regular tumour-specific peer review meetings (TPRMs) were established by our group during 2016. A dedicated Quality Assurance Radiation Therapist (QART) was employed in 2018 to co-ordinate the meetings and for each patient, complete the Peer Review Audit Tool (PRAT) of the Royal Australian and New Zealand College of Radiologists (RANZCR). The aim of the current quality assurance study was to investigate the impact of the TPRMs and appointment of the QART on compliance to relevant RANZCR Radiation Oncology Practice Standards (ROPS).

METHODS

Tumour-specific peer review meetings for eight tumour sites were assessed across our group's three hospitals from January 2017 to December 2019. Data from meetings were collected using the PRAT or from paper-based minutes and assessed against four ROPS (ROPS 3, 4, 8 and 9). Compliance with each of the four standards was measured by presence of the required documentation and presentation at TPRM, as recorded by the PRAT.

RESULTS

There was an increase in the overall number of peer review cases audited from 173 in the 2017 calendar year to 469 in 2018 and 619 in 2019, representing 7%, 18% and 22% of all treatment courses started during these years, respectively. Staging was the most incompletely documented item across all years for audited patients. The request for radiation treatment plan modifications increased year-on-year: modifications were requested for 5% of plans in 2017 (8/172), 18% in 2018 (81/452) and 19% (119/619) in 2019.

CONCLUSION

This study has shown that an increase in the number of cases for peer-review audit corresponded to the QART-facilitated TPRMs. Application of the PRAT has identified radiation treatment plan modifications that would otherwise go undetected and without opportunity to improve the quality of patients' treatment or avoid harm.

Analysis of Virtual Versus In-Person Prospective Peer Review Workflow in a Multisite Academic Radiation Oncology Department

PURPOSE

In radiation oncology, peer review is a process where subjective treatment planning decisions are assessed by those independent of the prescribing physician. Before March 2020, all peer review sessions occurred in person; however due to the COVID-19 pandemic, the peer-review workflow was transitioned from in-person to virtual. We sought to assess any differences between virtual versus in-person prospective peer review.

METHODS AND MATERIALS

Patients scheduled to receive nonemergent nonprocedural radiation therapy (RT) were presented daily at prospective peer-review before the start of RT administration. Planning software was used, with critical evaluation of several variables including treatment intent, contour definition, treatment target coverage, and risk to critical structures. A deviation was defined as any suggested plan revision.

RESULTS

In the study, 274 treatment plans evaluated in-person in 2017 to 2018 were compared with 195 plans evaluated virtually in 2021. There were significant differences in palliative intent (36% vs 22%; P = .002), but not in total time between simulation and the start of treatment (9.2 vs 10.0 days; P = .10). Overall deviations (8.0% in-person vs 2.6% virtual; P = .015) were significantly reduced in virtual peer review.

CONCLUSIONS

Prospective daily peer review of radiation oncology treatment plans can be performed virtually with similar timeliness of patient care compared with in-person peer review. A decrease in deviation rate in the virtual peer review setting will need to be further investigated to determine whether virtual workflow can be considered a standard of care.

A Comprehensive Analysis of a Prospective Multidisciplinary Peer Review Process Before Radiation Therapy Simulation

PURPOSE

Although peer review in radiation oncology (RO) has been recommended to improve quality of care, an analysis of modifications resulting from an RO multidisciplinary presimulation standardized review process has yet to be empirically demonstrated.

METHODS AND MATERIALS

A standardized simulation directive was used for patients undergoing simulation for external beam radiation therapy at a single tertiary care institution. The simulation directives were presented, and all aspects were reviewed by representatives from key RO disciplines. Modifications to the original directives were prospectively captured in a quality improvement registry. Association between key variables and the incidence of modifications were performed using Fisher exact test and t test.

RESULTS

A registry of 500 consecutive simulations for patients undergoing radiation therapy was reviewed. A median of 105 simulations occurred per month. All simulation directives were entered by a physician a median of 3 days before simulation (range, 1-76 days). The treatment intent was curative for 269 patients (53.8%), palliative for 203 patients (40.6%), and benign for 3 patients (0.6%). Twenty-five (5%) patients did not have a treatment intent selected. Based on RO multidisciplinary review, 105 directives (21%) were modified from the original intent, with 29 (5.8%) requiring more than 1 modification. A total of 149 modifications were made and categorized as changes to patient positioning and immobilization (n = 100, 20%), treatment site and care path (n = 34, 6.8%), simulation coordination activities (n = 6, 1.2%), and treatment technique and planning instructions (n = 9, 1.8%). A higher proportion of modifications occurred at the time of multidisciplinary review in patients receiving more complex treatments (intensity modulated radiation therapy/stereotactic radiosurgery/stereotactic body radiation therapy [IMRT/SRS/SBRT] vs 3-dimensional radiation therapy [3DCRT] radiation therapy, 25% vs 16%, P < .025).

CONCLUSIONS

Given the complexity of radiation therapy simulation, standardization of directives with prospective RO multidisciplinary presimulation peer review is critical to optimizing department processes and reducing errors. Approximately 1 in 5 patients benefits from this peer review process, especially patients treated with IMRT/SRS/SBRT.

100% peer review in radiation oncology: is it feasible?

Purpose

Peer review has been proposed as a strategy to ensure patient safety and plan quality in radiation oncology. Despite its potential benefits, barriers commonly exist to its optimal implementation in daily clinical routine. Our purpose is to analyze peer-review process at our institution.

Methods and materials

Based on our group peer-review process, we quantified the rate of plan changes, time and resources needed for this process. Prospectively, data on cases presented at our institutional peer-review conference attended by physicians, resident physicians and physicists were collected. Items such as time to present per case, type of patient (adult or pediatric), treatment intent, dose, aimed technique, disease location and receipt of previous radiation were gathered. Cases were then analyzed to determine the rate of major change, minor change and plan rejection after presentation as well as the median time per session.

Results

Over a period of 4 weeks, 148 cases were reviewed. Median of attendants was six physicians, three in-training-physicians and one physicist. Median time per session was 38 (4–72) minutes. 59.5% of cases presented in 1–4 min, 32.4% in 5–9 min and 8.1% in ≥ 10 min. 79.1% of cases were accepted without changes, 11.5% with minor changes, 6% with major changes and 3.4% were rejected with indication of new presentation. Most frequent reason of change was contouring corrections (53.8%) followed by dose or fractionation (26.9%).

Conclusion

Everyday group consensus peer review is an efficient manner to recollect clinical and technical data of cases presented to ensure quality radiation care before initiation of treatment as well as ensuring department quality in a feedback team environment. This model is feasible within the normal operation of every radiation oncology Department.