Impact of Multi-Institutional Prospective Peer Review on Target and Organ-at-Risk Delineation in Radiation Therapy

Systematic Implementation of Effective Quality Assurance Processes for the Assessment of Radiation Target Volumes in Head and Neck Cancer

PURPOSE

Significant heterogeneity exists in clinical quality assurance (QA) practices within radiation oncology departments, with most chart rounds lacking prospective peer-reviewed contour evaluation. This has the potential to significantly affect patient outcomes, particularly for head and neck cancers (HNC) given the large variance in target volume delineation. With this understanding, we incorporated a prospective systematic peer contour-review process into our workflow for all patients with HNC. This study aims to assess the effectiveness of implementing prospective peer review into practice for our National Cancer Institute Designated Cancer Center and to report factors associated with contour modifications.

METHODS AND MATERIALS

Starting in November 2020, our department adopted a systematic QA process with real-time metrics, in which contours for all patients with HNC treated with radiation therapy were prospectively peer reviewed and graded. Contours were graded with green (unnecessary), yellow (minor), or red (major) colors based on the degree of peer-recommended modifications. Contours from November 2020 through September 2021 were included for analysis.

RESULTS

Three hundred sixty contours were included. Contour grades were made up of 89.7% green, 8.9% yellow, and 1.4% red grades. Physicians with >12 months of clinical experience were less likely to have contour changes requested than those with <12 months (8.3% vs 40.9%; P < .001). Contour grades were significantly associated with physician case load, with physicians presenting more than the median number of 50 cases having significantly less modifications requested than those presenting <50 (6.7% vs 13.3%; P = .013). Physicians working with a resident or fellow were less likely to have contour changes requested than those without a trainee (5.2% vs 12.6%; P = .039). Frequency of major modification requests significantly decreased over time after adoption of prospective peer contour review, with no red grades occurring >6 months after adoption.

CONCLUSIONS

This study highlights the importance of prospective peer contour-review implementation into systematic clinical QA processes for HNC. Physician experience proved to be the highest predictor of approved contours. A growth curve was demonstrated, with major modifications declining after prospective contour review implementation. Even within a high-volume academic practice with subspecialist attendings, >10% of patients had contour changes made as a direct result of prospective peer review.

Quality and Safety Considerations in Image Guided Radiation Therapy: An ASTRO Safety White Paper Update

PURPOSE

This updated report on image guided radiation therapy (IGRT) is part of a series of consensus-based white papers previously published by the American Society for Radiation Oncology addressing patient safety. Since the first white papers were published, IGRT technology and procedures have progressed significantly such that these procedures are now more commonly used. The use of IGRT has now extended beyond high-precision treatments, such as stereotactic radiosurgery and stereotactic body radiation therapy, and into routine clinical practice for many treatment techniques and anatomic sites. Therefore, quality and patient safety considerations for these techniques remain an important area of focus.

METHODS AND MATERIALS

The American Society for Radiation Oncology convened an interdisciplinary task force to assess the original IGRT white paper and update content where appropriate. Recommendations were created using a consensus-building methodology, and task force members indicated their level of agreement based on a 5-point Likert scale from "strongly agree" to "strongly disagree." A prespecified threshold of ≥75% of raters who selected "strongly agree" or "agree" indicated consensus.

SUMMARY

This IGRT white paper builds on the previous version and uses other guidance documents to primarily focus on processes related to quality and safety. IGRT requires an interdisciplinary team-based approach, staffed by appropriately trained specialists, as well as significant personnel resources, specialized technology, and implementation time. A thorough feasibility analysis of resources is required to achieve the clinical and technical goals and should be discussed with all personnel before undertaking new imaging techniques. A comprehensive quality-assurance program must be developed, using established guidance, to ensure IGRT is performed in a safe and effective manner. As IGRT technologies continue to improve or emerge, existing practice guidelines should be reviewed or updated regularly according to the latest American Association of Physicists in Medicine Task Group reports or guidelines. Patient safety in the application of IGRT is everyone's responsibility, and professional organizations, regulators, vendors, and end-users must demonstrate a clear commitment to working together to ensure the highest levels of safety.

Assessing initial plan check efficacy using TG 275 failure modes and incident reporting

Plan checks are important components of a robust quality assurance (QA) program. Recently, the American Association of Physicists in Medicine (AAPM) published two reports concerning plan and chart checking, Task Group (TG) 275 and Medical Physics Practice Guideline (MPPG) 11.A. The purpose of the current study was to crosswalk initial plan check failure modes revealed in TG 275 against our institutional QA program and local incident reporting data. Ten physicists reviewed 46 high‐risk failure modes reported in Table S1.A.i of the TG 275 report. The committee identified steps in our planning process which sufficiently checked each failure mode. Failure modes that were not covered were noted for follow‐up. A multidisciplinary committee reviewed the narratives of 1599 locally‐reported incidents in our Radiation Oncology Incident Learning System (ROILS) database and categorized each into the high‐risk TG 275 failure modes. We found that over half of the 46 high‐risk failure modes, six of which were top‐ten failure modes, were covered in part by daily contouring peer‐review rounds, upstream of the traditional initial plan check. Five failure modes were not adequately covered, three of which concerned pregnancy, pacemakers, and prior dose. Of the 1599 incidents analyzed, 710 were germane to the initial plan check, 23.4% of which concerned missing pregnancy attestations. Most, however, were caught prior to CT simulation (98.8%). Physics review and initial plan check were the least efficacious checks, with error detection rates of 31.8% and 31.3%, respectively, for some failure modes. Our QA process that includes daily contouring rounds resulted in increased upstream error detection. This work has led to several initiatives in the department, including increased automation and enhancement of several policies and procedures. With TG 275 and MPPG 11.A as a guide, we strongly recommend that departments consider an internal chart checking policy and procedure review.

Radiation oncology peer review in Australia and New Zealand

Peer review is a part of high quality care within radiation oncology, designed to achieve the best outcomes for patients. We discuss the importance of and evidence for peer review in clinical practice. The Royal Australia and New Zealand College of Radiologists (RANZCR) has evolved a Peer Review Assessment Tool (PRAT) since 1999. We report the results of a RANZCR faculty survey conducted in radiation oncology facilities across Australia and New Zealand to guide the 2019 PRAT revision process, and discuss the development and implementation of the 2019 PRAT. Peer-review processes are now mandated as a component of Australian and International Quality Standards. Several practical recommendations might address challenges for effective implementation of peer review process in routine clinical practice. This includes prioritising tumour sites and treatment techniques for peer review within the time and resources constraints of each institution, improving resource allocation, ensuring optimal timing and duration for peer review meetings, and adopting multi-centre virtual peer review meeting where necessary.

Peer Review of Head and Neck Cancer Planning Target Volumes in Radiation Oncology

Purpose

Radiation treatment plans undergo peer review during chart rounds, but changes to treatment volumes would require replanning. Our group implemented weekly head and neck cancer “volume rounds” to peer review all target volumes for head and neck cancer before radiation therapy (RT) planning and chart rounds.

Methods and Materials

We analyzed modifications made to planning target volumes (PTVs) at volume rounds for consecutive nonproton head and neck cancer cases from May 2020 to May 2021. Nine head and neck radiation oncologists participated in weekly volume rounds during this time. Recommendations were categorized as no changes, minor changes, major changes, additional workup (eg, biopsy or imaging), and consultation or tumor board discussion needed before the start of RT. Minor changes to PTVs generally did not require a second review before treatment planning while major changes did.

Results

PTVs for 511 cases involving 432 patients underwent peer review and 298 (58.3%) of these cases did not require any modifications before treatment planning. Minor and major changes were recommended in 75 (14.7%) and 86 (16.8%) cases, respectively. Forty-five (8.8%) cases were recommended to have additional workup and 23 (4.5%) required additional consultation with nonradiation surgeons or medical oncologists. Of the 45 cases that were recommended for additional workup, 40 underwent biopsy or imaging. Positive findings on imaging or biopsy occurred in 13 patients, leading to a significant change in management, including 4 patients who underwent additional surgery after positive findings before the start of RT.

Conclusions

Prospective peer review during head and neck cancer volume rounds led to frequent minor and major alterations to PTVs. Significant changes in the overall treatment plan, such as additional surgery before start of RT, occurred in a minority of patients.

ProCaLung - Peer review in stage III, mediastinal node-positive, non-small-cell lung cancer: How to benchmark clinical practice of nodal target volume definition and delineation in Belgium☆

BACKGROUND AND PURPOSE

The Quality Assurance project for stage III non-small cell lung cancer radiotherapy ProCaLung performed a multicentric two-step exercise evaluating mediastinal nodal Target Volume Definition and Delineation (TVD) variability and the opportunity for standardization. The TVD variability before and after providing detailed guidelines and the value of qualitative contour reviewing before applying quantitative measures were investigated.

MATERIALS AND METHODS

The case of a patient with stage III NSCLC and involved mediastinal lymph nodes was used as a basis for this study. Twenty-two radiation oncologists from nineteen centers in Belgium and Luxembourg participated in at least one of two phases of the project (before and after introduction of ProCaLung contouring guidelines). The resulting thirty-three mediastinal nodal GTV and CTV contours were then evaluated using a qualitative-before-quantitative (QBQ) approach. First, a qualitative analysis was performed, evaluating adherence to most recent guidelines. From this, a list of observed deviations was created and these were used to evaluate contour conformity. The second analysis was quantitative, using overlap and surface distance measures to compare contours within qualitative groups and between phases. A 'most robust' reference volume for these analyses was created using the STAPLE-algorithm and an averaging method.

RESULTS

Five GTV and seven CTV qualitative groups were identified. Second step contours were more often in higher-conformity groups (p = 0.012 for GTV and p = 0.024 for CTV). Median Residual Mean Square Distances improved from 2.34 mm to 1.36 mm for GTV (p = 0.01) and from 4.53 mm to 1.58 mm for CTV (p < 0.0001). Median Dice coefficients increased from 0.81 to 0.84 for GTV (p = 0.07) and from 0.82 to 0.89 for CTV (p ≤ 0.001). Using HC-contours only to generate references translated in more robust quantitative evaluations.

CONCLUSION

Variability of mediastinal nodal TVD was reduced after providing the ProCaLung consensus guidelines. A qualitative review was essential for providing meaningful quantitative measures.

The impact of radiation therapist-led structured peer review meetings on compliance to Radiation Oncology Practice Standards

INTRODUCTION

Regular tumour-specific peer review meetings (TPRMs) were established by our group during 2016. A dedicated Quality Assurance Radiation Therapist (QART) was employed in 2018 to co-ordinate the meetings and for each patient, complete the Peer Review Audit Tool (PRAT) of the Royal Australian and New Zealand College of Radiologists (RANZCR). The aim of the current quality assurance study was to investigate the impact of the TPRMs and appointment of the QART on compliance to relevant RANZCR Radiation Oncology Practice Standards (ROPS).

METHODS

Tumour-specific peer review meetings for eight tumour sites were assessed across our group's three hospitals from January 2017 to December 2019. Data from meetings were collected using the PRAT or from paper-based minutes and assessed against four ROPS (ROPS 3, 4, 8 and 9). Compliance with each of the four standards was measured by presence of the required documentation and presentation at TPRM, as recorded by the PRAT.

RESULTS

There was an increase in the overall number of peer review cases audited from 173 in the 2017 calendar year to 469 in 2018 and 619 in 2019, representing 7%, 18% and 22% of all treatment courses started during these years, respectively. Staging was the most incompletely documented item across all years for audited patients. The request for radiation treatment plan modifications increased year-on-year: modifications were requested for 5% of plans in 2017 (8/172), 18% in 2018 (81/452) and 19% (119/619) in 2019.

CONCLUSION

This study has shown that an increase in the number of cases for peer-review audit corresponded to the QART-facilitated TPRMs. Application of the PRAT has identified radiation treatment plan modifications that would otherwise go undetected and without opportunity to improve the quality of patients' treatment or avoid harm.

Estimation of planning organ at risk volumes for ocular structures in dogs undergoing three-dimensional image-guided periocular radiotherapy with rigid bite block immobilization

Planning organ at risk volume (PRV) estimates have been reported as methods for sparing organs at risk (OARs) during radiation therapy, especially for hypofractioned and/or dose‐escalated protocols. The objectives of this retrospective, analytical, observational study were to evaluate peri‐ocular OAR shifts and derive PRVs in a sample of dogs undergoing radiation therapy for periocular tumors. Inclusion criteria were as follows: dogs irradiated for periocular tumors, with 3D‐image‐guidance and at least four cone‐beam CTs (CBCTs) used for position verification, and positioning in a rigid bite block immobilization device. Peri‐ocular OARs were contoured on each CBCT and the systematic and random error of the shifts in relation to the planning CT position computed. The formula 1.3×Σ+0.5xσ was used to generate a PRV of each OAR in the dorsoventral, mediolateral, and craniocaudal axis. A total of 30 dogs were sampled, with 450 OARs contoured, and 2145 shifts assessed. The PRV expansion was qualitatively different for each organ (1‐4 mm for the dorsoventral and 1‐2 mm for the mediolateral and craniocaudal axes). Maximal PRV expansion was ≤4 mm and directional for the majority; most pronounced for corneas and retinas. Findings from the current study may help improve awareness of and minimization of radiation dose in peri‐ocular OARs for future canine patients. Because some OARs were difficult to visualize on CBCTs and/ or to delineate on the planning CT, authors recommend that PRV estimates be institution‐specific and applied with caution.

VA-Radiation Oncology Quality Surveillance Program

PURPOSE

We sought to develop a quality surveillance program for approximately 15,000 US veterans treated at the 40 radiation oncology facilities at the Veterans Affairs (VA) hospitals each year.

METHODS AND MATERIALS

State-of-the-art technologies were used with the goal to improve clinical outcomes while providing the best possible care to veterans. To measure quality of care and service rendered to veterans, the Veterans Health Administration established the VA Radiation Oncology Quality Surveillance program. The program carries forward the American College of Radiology Quality Research in Radiation Oncology project methodology of assessing the wide variation in practice pattern and quality of care in radiation therapy by developing clinical quality measures (QM) used as quality indices. These QM data provide feedback to physicians by identifying areas for improvement in the process of care and identifying the adoption of evidence-based recommendations for radiation therapy.

RESULTS

Disease-site expert panels organized by the American Society for Radiation Oncology (ASTRO) defined quality measures and established scoring criteria for prostate cancer (intermediate and high risk), non-small cell lung cancer (IIIA/B stage), and small cell lung cancer (limited stage) case presentations. Data elements for 1567 patients from the 40 VA radiation oncology practices were abstracted from the electronic medical records and treatment management and planning systems. Overall, the 1567 assessed cases passed 82.4% of all QM. Pass rates for QM for the 773 lung and 794 prostate cases were 78.0% and 87.2%, respectively. Marked variations, however, were noted in the pass rates for QM when tumor site, clinical pathway, or performing centers were separately examined.

CONCLUSIONS

The peer-review protected VA-Radiation Oncology Surveillance program based on clinical quality measures allows providers to compare their clinical practice to peers and to make meaningful adjustments in their personal patterns of care unobtrusively.

Results of a multicenter intensity modulated radiation therapy treatment planning comparison study for a sample prostate cancer case

PURPOSE

To determine the influence of different medical physicists, photon energies, treatment planning systems and treatment machines on the resulting external beam radiotherapy dose distribution for a sample prostate cancer case.

METHODS

A pre-contoured computed tomography (CT) dataset containing planning target volume 1 (PTV1) prostate and seminal vesicles (single dose [SD] 1.8 Gy, total dose [TD] 59.4 Gy), PTV2 prostate (simultaneously integrated boost [SIB], SD 2.0 Gy, TD 66 Gy), PTV3 prostate and seminal vesicles approach (SD 1.8 Gy, TD 73.8 Gy/80.4 Gy SIB) as well as organs at risk (OAR: rectum, bladder, femoral heads, bowel, anus) was offered to the members of the task group IMRT (intensity-modulated radiation therapy) of the German Society for Medical Physics. The purpose was to calculate one combined treatment plan (TP) for PTV1 and PTV2, as well as a separate one for PTV3. Dose volume histograms (DVH), different dose values, conformity index (CI), homogeneity index (HI), gradient index (GI) and a new "better than average score" were used to analyse the dose distributions.

RESULTS

Altogether 44 institutions took part in this study and submitted acceptable dose distributions for the PTVs. However, there were statistically significant differences, especially for the doses administered to the OAR, such as rectum, bladder and femoral heads. Differences between the treatment plans were not easily detectable by visual inspection of the isodose distribution. Dose maxima may occur outside the PTV. Even though scoring indices are already published, the new "better than average score" was needed to identify a plan that minimises dose to all OAR simultaneously.

CONCLUSION

Different medical physicists or dosimetrists, photon energies, treatment planning systems, and treatment machines have an impact on the resulting dose distribution. However, the differences only become apparent when comparing DVH, analysing dose values, comparing CI, HI, GI, as well as reviewing the dose distribution in every single plane. A new score was introduced to identify treatment plans that simultaneously deliver a low dose to all OAR. Such inter- and intra-institutional comparison studies are needed to explore different treatment planning strategies; however, there is still no automatic solution for an "optimal" treatment plan.